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ABSTRACT 

The American Industry Project represents one attempt 
to resolve the problem of providing career orientation for young 
people in a way which would: (1) avoid premature tracking, (2) help 
youth to identify the numerous career options open to them, (3) 
furnish base knowledges and coping strategies which help youth to 
adapt to change, and (4) establish a delivery system for teachers of 
the new program. Part I of this final report offers a narrative 
overview of the total project, addressing itself specifically to the 
achievement of the objectives of the project as stated in the 
original proposal, while Part II presents an analysis of the data 
collected and a description of the evaluation process used in 
developing and testing the curriculum materials. The Project 
developed a conceptual structure of the knowledges used in industry, 
and three secondary courses based on this structure were designed. 
Analysis of the data collected from the Project's achievement test 
revealed that approximately 40 percent of the students taught by 
instructors who followed the instructor's guide or used its 
objectives and content to at least a moderate degree achieved 
significantly higher scores than students in the control group. These 
and other results are discussed, and several project materials are 
included in the separate appendix section. (Authors/SB) 
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Abstract 



Part I: Overview of the Project 

The American Industry Project represents one attempt to resolve the problem of providing career 
orientation for young people in a way which would avoid premature tracking; help youth to identify the 
numerous career options open to them; furnish base knowledge:; and coping strategies which would help 
youth to be adaptable to change; and establish a delivery system for teachers of the new program.. 

Project efforts were aimed to meet the following objectives: 

To complete the development of an articulated program of American Industry at 
three levels leading into vocational preparation. 

To establish, at Stout State University, a resource center for the program of Ameri- 
can Industry. 

To complete the preparation of teaching materials for the upper two levels of Ameri- 
can Industry and refinement of the first year's materials. 

To carry on a series of institutes for the preparation of in-service teachers who will 
introduce the program of American Industry into their schools. 

To carry on a pilot teacher education program culminating in a recommended curri- 
culum for the undergraduate preparation of teachers of American Industry. 

To identify research problems associated with the development and the implementa- 
tion of the articulated levels of American Industry and to encourage research action 
seeking answers to these problems. 

Impacts of the project effort on the institution included significant changes in the structure of the 
university; namely changes in departmental organization in the School of Industry and Technology and 
the implementation of a Center for Improvement of Learning and Instruction a mechanism for intro- 
ducing innovation. Significant growth was also noted in the personnel involved in the project. 



Objective A: 



Objective B: 



Objective C: 



Objective D: 



Objective E: 



Objective F: 
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Overview of the Project 
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Eugene R. F. Flug 
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American Industry Project 



CHAPTER I 



ACHIEVING THE OBJECTIVES OF THE PROJECT 



Introduction 



This volume of the report consists of two parts. Part I offers a narrative overview of the total project 
in addressing itself specifically to the achievement of the objectives of the project stated in the original 
proposal. Part II presents analyses of the data collected and a description of the evaluation process used in 
developing and testing the curriculum materials. Each part may be read separately to gain different perspec- 
tives of the work. It should be noted that common references are made to other papers, reports, and forms 
which have been bound separately in an appendix to make for easier use by the reader. 

The American Industry Project represents one attempt to resolve the problem of providing career 
orientation for young people in a way which would avoid premature tracking; help youth to identify the 
numerous career options open to them; furnish base knowledges and coping strategies which would help 
youth to be adaptable to change; and establish a delivery system for teachers of the new program. (See 
Appendix P for the transcript of a more detailed presentation cf the rationale underlying the project.) 



The Objectives of the Projeuc 



Objective A: To complete the development of an articulated program of American Industry at three 
levels leading into vocational preparation. 

One year of development had been completed under a grant from the Pord Foundation prior to federal 
funding. In that time ten secondary school teachers had participated in a summer's work where they wrestled 
with the task of stating objectives, developing learning activities for each objective, and suggesting resource 
materials for the teacher. These teacher developed materials were put together into fourteen volumes for use 
by each teacher in designing his own course of study. Conferences during the year provided for enough inter- 
change to demonstrate that success was mixed and problems weie manifold. Federal funding made it possible 
to gather a staff to carry on a more orderly and continuing development of materials and design a model 
program to be implemented by all of our co jr grating teachers. (A more definitive presentation of the devel- 
opment of materials may be seen in Appendix ' ) 

A very real setback was experienced in the transition to federal funding. Although we had been in- 
formed that thd project would be funded under section 4C of the Vocational Education Act of 1963, there 
was a significant dbiay in contracting. A preliminary letter of credit authorized expenditure of a minimum 
amount of funds. However, it was November before we received a contract, thus it was necessary for the 
university to reluctantly permit us to employ the additional staff needed before the contract had been 
received. This greatly limited the time available to recruit staff who would be able to step right into a 
productive role of curriculum building. Y'v were fortunate in locating people who were eager to learn and 
not afraid of hard work. 
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While some progress was made in the first year, considerable time was consumed in orienting new 
staff to their tasks and generally gearing up to a new level of operation. Also, had wa been free to offer 
premium salaries to experienced professionals, we might have been able to reduce our start up time. 

A conscious decision was made midway through the project to concentrate our attention on the first 
level course in the program. Since this was the first contact students would have with American Industry, we 
felt it deserved this emphasis. The first level was aimed to develop an initial frame of reference, or global 
construct, of American Industry and it was realized, that for many students, it might be their only exposure. 

It was felt to be important that this level represent the best we could produce with the time and the funds 
available. 

Materials were produced and tested at all three levels, but it was possible for us to pilot all three levels 
together in only one school. We have produced enough evidence, however, to feel that the basic approach is 
sound and could provide an excellent route to vocational preparation. There is some indication that the first 
level course alone provides a good base for vocational choice for some young people. Consequently, the third 
level course, which incorporates independent research within one of the major concept areas, might provide 
a particularly appropriate step toward the four year degree program for those so inclined. 

Objective B: To establish, at Stout State University, a resource center for the program of American 
Industry. 

The primary purpose for the inclusion of this objective was to provide for the immediate availability of 
resource materials which could be of use in the day-to-day tasks of curriculum development. A few sets of 
multiple copies of certain basic references were acquired for use with workshops and summer institutes. 

Both content and process references were collected. It was found that there were no comprehensive resources 
concerned with industry as conceived by the project staff, so efforts were made to search by the thirteen 
concept areas represented in our knowledge structure. (See Figure 3 in Appendix B-5 for the conceptual 
structure derived in the study.) 

A continuing search was also conducted in our campus library and suggestions for new acquisitions 
were supplied by project staff to build up the main library in these concept areas. 

Within the project resource center, special - nphasls was placed upon acquiring those materials not 
normally available through our campus library. These included some curriculum packages, audio tapes, film- 
* rips, specialized journals, and numerous house publications from industry and newsletters from various 
agencies and projects. A valuable aspect of the resource center was its organization around the thirteen con- 
cept areas. Much time was saved thereby in accessing materials. Over five thousand separate pieces have 
been catalogued and used. 

Assistance was provided by library staff in all . bases of the development of the resource center. The 
university is moving to consolidate its library and audio-visual services into a learning resource center. A 
newly established educational curriculum materials center may eventually become the repository for the 
American Industry resource center. 

Because of its unique organization about the thirteen concept areas, the resource center has also come 
to be utilized rather extensively by studeois in the American Industry undergraduate degree program and in 
the graduate concentration in American Industry. 
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O bjective C: To complete the preparation of teaching material far the upper twr. levels of American 
industry and refinement of the first year's materials. 

At the completion of the project, the first level materials had undergone three cycles of deveiop-tryoui 
revise, and were considered ready for dissemination. This instructional package consisted of an inst. actor's 
guide, film catalog, fifty-two overhead transparencies, a set of transparency masters, three slide series, one 
filmstrip, and a set of seven student booklets. The package is not included as a part of this report because 
arrangements have been made for a joint copyright with the Office of Education, and a publisher is being 
sought for commercial distribution. One complete set is being furnished separately from this report as an ex- 
hibit for the informational use of the Office of Education staff. 

A twenty minute sound color film entitled, "Introduction to American Industry", was produced entirely 
with university funds by the Audio-Visual Department at Stout and is an integral part of several activities in the 
course. 

We were unable to carry through the refinement of the advanced course materials for levels II and III as 
we had hoped since the funding necessary to accomplish this task was not available. Plans are presently being 
implemented to continue the developmental work within the School of Industry uid Technology at Stout. 
Single exhibit copies of these materials are also being submitted along with this report. Level II and III in- 
structor guides will become a part of the total package to be copyrighted. 

It is important to point out here that the university provided most generous support for this project, 
substantially exceeding minimum required matching funds. 

Objective D: To carry on a series of institutes for the preparation of in-service teacher? who will intro- 
duce the program of American Industry into their schools. 

in the first two years of federal funding, participating teachers were br< ught onto the campus to revise 
and develop materials for the following year. New teachers were added in each of the two years as we tried to 
get the representation of schools we desired for pilot testing the materials. It was necessary, because of re- 
stricted funds in the last two years, to shift to other means of preparing new teachers. A selected few partici- 
pating teachers were employed to work on revising materials or to assist in preparing new teachers through the 
regular graduate program. Also, the Education Professions Development Act made it possible to hold two 
successive institutes as a part of our dissemination strategy. It was not possible, however, to involve these new 
teachers as fully in the project as were the participating teachers. 

Extension courses were developed and are now being deployed about the state during the regular acade- 
mic year. 

Objective E: To carry on a pilot teacher education program culminating in a recommended curriculum 
for the undergraduate preparation of teachers of American Industry. 

The assignment to fulfill this objective was made to Dr. Lorry Sedgwick, who wa.' employed in the first 
year of the project under federal funds. In discussions with President Michaels, we were urged to free our 
thinking and look for new solutions without restriction to established forms. Accordingly, Dr. Sedgwick spent 
the better part of the first year in building a model of the effective teacher as supported by research or a signi- 
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ficant body of opinion. (Appendix Q presents the Teacher Model used to guide the development of the 
teacher education program.) 



The essential characteristics of the four year degree program ware its breadth of course experiences, the 
early introduction to teaching, and the integration of professional teacher education content into the total 
four year program. (Dr. Sedgwick's paper describing the program is shown in Appendix R, and the compe- 
tencies expected are listed in Appendix S.) Recognition for outstanding achievement in innovative teacher 
education programs was given Stout for the American Industry program by the American Association of 
Colleges for Teacher Education in 1968. 

The American Industry degree program was accepted as a regular offering of the university well before 
the termination of the federal funding period. (Appendix T shows the degree requirements for the American 
Industry major. The requirements for a dual major in American Industry/Industrial Arts are shown in Ap- 
pendix U.) 

Ofejoctiye F; To identify researc h problems associated with the development and tha implementation 
of the artic ulated levels of American industry and to encourage research action seeking answers to these 
problems. 

Achievement of this objective was a natural concomitant of the ongoing project. Tha teachers partici- 
pating in the project involved themselves in graduate degree programs and availed themselves of the oppor- 
tunity to complete the research requirements with American Industry problems. One participant completed 
his doctor s degroe and is now teaching at a major universitv , and another will soon complete his doctorate. 
Four project staff members also completed their doctorates, incorporating work on American Industry pro- 
blems. Numerous g aduate students not directly involved in the work also produced related research. In 

many cases related research was conducted by those who visited the project at Stout or observed the field 
centers. 

Spinoff activities of many kinds resulted from project efforts. One important contribution was the 
introduction of micro-teaching to the Upper Midwest by project staff. (See Harlyn Misfeldt's paper in 
Appendix V.) Another was the establishment of the Center for Improvement of Learning and Instruction 
with the general responsibility of introducing and enabling innovation on campus. Growing interest in 
career education also makes it likely that a Center for Vocational Education will be established at Stout 
through joint action of the university and the Wisconsin Board for Vocational, Technical and Adult Education. 
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CHAPTER II 



EFFECTS OF THE AMERICAN INDUSTRY 
PROJECT UPON THE INSTITUTION 



Introduction 



The American Industry Project was the first large scale, federally funded project implemented in the 
Wisconsin State University System. It also marked the first significant venture for Stout State University into 
research and development. This effort was launched at the outset of one of the greatest growth periods in 
the history of both system and institution. Under these conditions one might expect that many decisions 
would need to be made without benefit of precedent. Many difficulties were encountered in attempting to 
perform functions for which university procedures had not been designed. To chronicle these problems 
would be worthy of a major work in itself but, in retrospect, it must be admitted that the project created a 
host of new problems for harried administrators who were overburdened and understaffed. The fact that 
difficulties were surmounted and positive effects resulted is the focus of this chapter. 



Influences on Program 

Perhaps the most significant institutional impact of the American Industry Project was bringing the 
status quo under searching scrutiny. Stout had long been a leader in industrial arts education, graduating 
more students in that field than any other university in the country. There is a natural tendency when dealing 
with relatively large numbers of students, to move cautiously in making curriculum changes because of the 
administrative readjustments necessary. Perhaps even more important, are the attitudinal problems behind 
the difficulty of shifting an orientation based in a long and proud tradition. The American Industry Project, 
with a mandate to find new solutions to career education problems, called into question not only existing 
practices, but the very bases on which they were predicated. 

Earnest efforts were made by the project staff to develop understanding and acceptance by sharing 
information and ideas with the university community. For many faculty, however, the new views were too 
radical a departure to accept. Skepticism and even outrqht resistance were not uncommon amoung the 
early attitudes faced. Consequently, it was much easier for project staff to focus on the challenges implicit 
in the curriculum tasks under way than to devote efforts toward cultivating scoffers and obstructionists. A 
common observation among project staff in the second year of the project was "People are more receptive 
to American Industry the farther away we get from Stout." 

This early management error was rectified in the third year with increasing attempts to involve 
university personnel in activities related to the project. Faculty were asked to critically evaluate materials 
and to visit participating centers. Key staff were invited into workshops with participating teachers and 
other faculty were asked to contribute to the dialogue generated in the professional teacher education's 
new interdisciplinary seminar. These and other efforts stimulated new discussions of philosophical posi- 
tions and ultimately resulted in the establishment of the dual major in American Industry/Industrial Arts. 
(See Appendix U) 
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The turning point in legitimizing American Industry on the campus came with the support of Dean 
Herbert Anderson of the School of Industry and Technology. Evaluation of the substance of the Ameri- 
can Industry program persuaded him of the promise inherent in the conceptual approach. Under his 
steady, determined leadership it is expected that American Industry will grow in influence and acceptance. 
This is not to say that all aspects of the program now meet with complete acceptance. On the contrary 
criticism is still evident but it has shifted to a healthy focus on implementation and one finds that the ’ 
questioning now includes ways in which campus programs can assimilate the best ideas of both innovation 
and tradition. Several basic courses of the American Industry program at the university level have been 
accepted as requirements for all incoming freshmen in the industrial arts program. A movement is under 
way to insure that all future industrial arts graduates of Stout will be capable of teaching at least the first 
level of American Industry in the intermediate and secondary schools. 



Influences on University Structure and Personnel 

Consideration of the American Industry Project's conceptual analysis of the knowledges of industry 
have led the School of Industry and Technology at Stout to accept responsibilty for maintaining *he 
currency of that structure. The structure is expected to serve as the basis for curriculum development 
within that school. Acceptance of the conceptual orientation has led to a re examination of the depart- 
mental structure and consideration is now being given to reorganizing the departments. The reorganization 
is aimed to better affix responsibility for concept areas embodied in the knowledge structure. 

Several years ago mention was made to the writer by an official of a major funding agency, that 
few of the projects they had funded had had much of an effect on existing practices. The most visible 
effect of their funding wss to be seen in the professional and personal growth of the staff involved in 
the projects. This kind of growth was also seen in American Industry Project staff. 

Dr. Face, one of the original co-directors of the project was appointed at the end of the project's 
second year, to half time responsibility as Assistant Dean of the Graduate School. In the fourth year 
he was appointed full time to Assistant to the Vice President for Academic Affairs and Curriculum 
Coordinator. At that time the writer was named Assistant to the President and Director of the Center 
for Improvement of Learning and Instruction, with responsibility for introducing innovation into the 
university. Dr. Nelson, Research Specialist on the project, has moved into a key role in the Canter of 
providing research and evaluation consultation to the university and to the state vocational system 
Harlyn Misfeldt, Coordinator of Professional Teacher Education on the project was recently elected to a 
similar position within the School of Education. Dr. Sedgwick, Director of the project's professional 
teacher education program provides 'eadership and consultation on individur.iited instruction as a staff 
member of the School of Education. 

William Daehling, Audio-Visual Specialist on the project left to complete his doctorate and is now 
a dean at Weber College. Utah. Richard Gebhart. Curriculum Specialist on the project, also completed 
his doctorate and is now on the staff of the Industrial Education Department. He is soon to initiate 
a statewide study of industrial arts funded by the Department of Public Instruction. Harold Olstad, 
Supervisor of Participating Teachers, has been re assigned as Program Director of tha American Industry 
major and is responsible for the continuing development of the campus based program. 
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Douglas Stallsmith and Charles Yost, Content Specialists in American Industry provide leadership 
in the School of Industry and Technology in the development of conceptually oriented laboratory courses. 
Mr. Stallsmith has also been called upon as a consultant to school systems in the design of flexible lab- 
oratories for industrial arts and related areas. All of the above have contributed to the design of the 
facilities to house the Center for Improvement of Learning and Instruction. 

Dwight Davis served for two years as Assistant to the Curriculum Specialist while a student at 
Stout. He has since completed advanced degree work and is Associate Dean for Curriculum at Joliet 
Junior College in Illinois. He is providing state and regional leadership in the development and testing 
of the comprehensive Joliet Curriculum Development Model funded by the state of Illinois. Other 
participants and students associated with the project in different ways have continued to show growth 
in their respective pursuits. 



The Center for Improvement of Learning and Instruction 

The opportunities for institutional growth which came to the university as a result of the American 
Industry Project, were recognized early. The writer was very much concerned that some means be 
found to provide for continuing this growth and increasing flexibility beyond the duration of the project. 
Consequently, a proposal was accepted by the President to incorporate into the university structure a 
unit which would serve as a mechanism for change. The Center for Improvement of Learning and In- 
struction was designed for the primary purpose of providing an outlet for faculty, staff, and student 
ideas which hold promise for improving university efforts to achieve the university's mission. Its emphasis 
is to be on applied problems in instruction, research, or service and it is intended to serve the entire 
university. 

The Center is intended "to get things started" and carried to the point where the new practice 
is being accepted into ongoing programs. Its operation is seen as one wherein there is the willingness 
tc try, to risk failure, to honestly say that the university encourages new thinking about the tasks that 
face us. The willingness to question the validity of the status quo in an effort to find better solutions 
is basic to its function. The Center is attached to the President's office because of its all-university 
function and can provide a vehicle for launching actions which the president deems in the best interests 
of the university in moving toward the achievement of long range goals. 

Two advisory groups are to be formed to guide the involvements of the Center. The first, rep- 
resentative in nature, is to aid in the formulation of policy and evaluation of operations of the Center. 

The second advisory group will be selected according to competence and interest in promoting improve- 
ment. When appropriate, this latter group will be called together to consider ideas presented by faculty 
or students. Its ‘function will be to suggest improvements in the proposal and plans for proceeding 
by providing unbiased, constructive criticism. 

Key features of the Center's operation involve provision of consultative assistance, clerical support, 
facilities and released time. Unassigned allocations are being sought so that faculty may be freed to 

work on projects supported by the Center. Office space and laboratory facilities will be made available 
as needed. 
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In September of 1972 the Center will become a functioning reality on Stout's campus. Figures 
1 and 2 on the following pages shows the layout of facilities being constructed within the new Applied 
Arts building. About 22,000 square feet has been assigned to the Center. The American Industry 
laboratory is intended to provide the utmost in flexibility as a demonstration of a model American 
Industry facility. Complete services including water, electricity, air pressure, vacuum lines and exhaust 
system are supplied from overhead. Equipment will be mounted on air pallets which, when hooked up 
to an air pressure line, will enable one to easily move the heaviest piece of equipment. This laboratory 
will provide ongoing instruction in American Industry but will also provide for pilot testing new con- 
figurations of equipment designed to optimize technical instruction. Two 'storage rooms, a finishing room 
and a dark room are integral parts of this 5000 square foot laboratory. 

Adjacent to the American Industry laboratory is the Learning Research Laboratory. Staff from 
the School of Education will utilize this facility for research in teaching methodology. Provisions for 
flexibility in this area include an overhead grid system which will make it possible to divide the ap- 
proximately 2200 square feet into modules of a great variety of sizes. Provisions can be made to enclose 
individualized learning areas and small group or large group areas by shifting movable panels which fit 
into the overhead grid system. The laboratory will be furnished with equipment which will make it 
possible to study a variety of applications of media to instruction. 

The third laboratory, of about 2400 square feet, houses equipment which will make it possible 
to fabricate new designs of equipment for technical instruction. As equip nent is developed it will be 
tested under instructional conditions in the adjacent laboratories. Flexibility is provided in this area 
through the use of a computer floor consisting of panels under which electrical and other services are 
provided to each panel. A concrete apron for heavier equipment surrounds the computer floor area. 
Supporting facilities for the laboratories include a seminar room, a media design and production area, 
a dark room, a clean drafting room for engineering artwork, a normal design and drafting room, and 
a large storage room. 

The remainder of the area assigned to the Center includes 17 offices, a small conference room, 
a stenographic pool, a storage and duplicating area, and an information processing and retrieval area. 

Three remote access computer terminals will be tied into our centralized computer facilities. A lounge 
area completes the complex. 

While there may be some question about broad-scale acceptance of the American Industry program 
in the intermediate and secondary schools there is no doubt that the effects on individuals involved have 
been considerable and that the impact on the institution has been pervasive and will prove to be long 
lasting. 
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Figure 2 
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Abstract 



Part II: Development, Implementation, and Evaluation 
of American Industry Secondary School Courses 



The American Industry Project grew out of the concern of a group of Stout State University profes- 
sors for the status of industrial arts and its future goals. The efforts and ideas of these men provided the 
basis for systematic development of the American Industry curriculum. Their decision to isolate the con- 
ceptual knowledges of industry and structure them in a logical manner was a key element in the design of 
this curricuU'r:. 

Data for the conceptual structure were obtained from written documents, interviews with people in 
industry and observation of industrial activities. The resulting structure was validated with a jury composed 
of educators and men in industry. It was the opinion of the jury that the structure of conceptual know- 
ledges was logical and representative of the knowledges used in industry. 

.Three secondary school courses based on this structure were designed and developed. The first course. 
Level I, was based on the major concepts in the structure. Level II, the second course in the sequence, 
was designed to go into greater depth through the study of selected sub-concepts in the structure. The 
last course. Level III, was developed to give the student an opportunity to select a problem and apply the 
concepts in the structure to it. 

The majority of the American Industry Project's Jat a caj .e from the experimental tryout of the Level 
I course. Time, budget, and student scheduling limitations reduced the amount of experience the Project 
had with Levels II and III. however, the results of the evaluation of Lbvel I provides considerable informa- 
tion for refining Level I and revising Levels II and III. 

Analysis of the data collected from the Project's achievement test revealed that approximately 40% 
of the American Industry Classes taught by instructors who followed the Project's instructor's guide or 
used its objectives and content to at least a moderate degree achieved significantly higher scores than the 
students in the control group. In an additional 12.5% of the comparisons, the F value approached signifi- 
cance with the American Industry students outscoring their counterparts in the control groups. In no 
comparison was there a significant difference in favor of the control group, and in only one of approxi- 
mately sixty comparisons did the mean for the control group exceed the mean for the American Industry 
group. 

In a study of interest: and attitudes conducted during the 1967*68 school year, the responses of the 
American Industry students gave evidence of a more positive attitude toward industry than those of the 
control group. Also, their responses indicated that the study of American Industry had an impact on their 
knowledge of occupational opportunities in industry and their interest in working in industry. A small- 
scale follow-up study with graduates who had taken the Level I American Industry course reinforced these 
findings. 

The results of the evaluation indicated that the study of American Industry had a measurable impact 
on the students' knowledge of industry and their interests in it. Data collected during the evaluation also 
gave a dear picture of the operating characteristics of the instructional materials developed for the courses. 
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PERMISSION TO USE COPYRIGHTED MATERIALS 

Permission was received to reproduce the following previously published mat- 
erials. Each work is identified by the page number on which it appears in this report. 



1. Figure 3, page 10. 

Reprinted from Journal of Systems Management, by Dr. Martin I. Taft, by 
permission of Dr. Martin I. Taft. Copyrighted 1971. Permission to reproduce 
this copyrighted material has been granted by Martin I. Taft to the Educational 
Resources Information Center (ERIC) and to the organization operating under 
contract with the Office of Education to reproduce ERIC documents. Reprod- 
uction by users of any copyrighted material contained in documents disseminated 
through the ERIC system requires permission of the copyright owner. 

2. SCOPE data on students on page 16. 

Reprinted from _SC_OPE: Four State Profile Grade Nine 1966 California. Illinois. 
Massachusetts. JNorth Carolina and. SCOPE: Four State Profile Grade Twelve 

1966 • California. Illinois. Ma«a*rhngptts. North Carolina f by Dale Tillery, Denis 
Donovan, and Barbara Sherman, by permission of SCOPE Project. Copyrighted, 
1966. Permission to reproduce this copyrighted material has been granted by 
Denis Donovan to the Educational Resources Information Center (ERIC) and to 
the organization operating under contract with the Office of Education to re- 
produce ERIC documents. Reproduction by users of any copyrighted material 
contained in documents disseminated through the ERIC system requires permis- 
sion f the copyright owner. 

3. Data on teachers’ salaries cr. pages 23 - 26. 

Reprinted from NEA Research Bulletin. Volumes: 43, 44, & no author 
listed, by permission of NEA Research Division. Copyrighted 1965, 1966, & 
1970. Permission to reproduce this copyrighted material has been granted by 
Beatrice C. Lee, Publications Editor, NEA Research Division to the Educational 
Resources Information Center (ERIC) and to the organization operating under 
contract writh the Office of Education to reproduce ERIC documents. Reprod- 
uction by users of any copyrighted material contained in documents disseminated 
through the ERIC system requires permission of the copyright owner. 
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4. Appendix 0: Instrument and key used to collect and analyze data on 

student attitudes. 

Reprinted from 44 Environmental Guidance of the Classroom Behavioral System'’ 
in Contemporary Research on Teachers Effectiveness (B. Biddle and W. EUena, 
Eds.), by Paul V. Gump, by permission of Paul V. Gump. Copyrighted 1964. 
Permission to reproduce this copyrighted material has been granted by Paul 
V. Gump to the Educational Resouices Information Center (ERIC) and to the 
organization operating under contract with the Office of Education to reprod- 
uce ERIC documents. Reproduction by users of any copyrighted material 
contained in documents disseminated through the ERIC system requires per- 
mission of the copyrighted owner. 



Non-Copy righted materials: 

The statistics reported in Table 4 on pages 2.24 and 2.25 from Studies of 
the American High School (Project Talent) are used with the permission of 
Dr. Marion F. Shaycoft, Association Director, Institute for Research in 
n Vacation, American Institutes for Research. 
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CHAPTER I 



DEVELOPING AND EVALUATING THE AMERICAN INDUSTRY 
SECONDARY SCHOOL CURRICULUM 



Introduction 



The contents of this report pertain to the activities undertaken to develop, refine, and eval- 
uate the American Industry secondary school curriculum. In the first part of this report which 
encompasses Chapters I, II, III, the reader will find a general presentation of the results of the 
American Industry evaluation. This presentation includes a brief history of the American Industry 
Project; a description of the American Industry secondary school curriculum, and a discussion of 
the evaluation results. 

In-depth reports of specific curriculum development and evaluation activities are presented in 
the appendixes. Readers interested in detailed and theoretical discussions should review the reports 
in the appendixes. Reference will be made to these reports in Chapters I, II, and III as they 
pertain to the topics under discussion. 



Genesis of the American Industry Project 



Man is continually seeking new methods and means to satisfy his basic needs and desires. 
This quest is reflected in our ever changing technology. Human muscle power is being replaced 
by chemical, electrical, and atomic energy. The craftsman has, in manv fields, been replaced by 
specialists and by specialized machinery. Man has become more and ...ure dependent upon others 
for the products he has to have to satisfy his needs. These are but a few examples of the 
changes being brought about by man's search for greater productivity. 

The factors ju*t noted have special importance for educational programs related to industry. 
With rapid changer in industry it is imperative that a curriculum such as industrial arts be up- 
dated to keep pace with the real world phenomena upon which it is based. This was the prob- 
lem that stimulated a group of professors at Stout State University to form a study group in 
1962 to review the content and direction of industrial arts. Out of their discussions and study 
grew the realization tha; there was a need for change in the industrial arts curriculum of the ra. 

This study group made two major decisions in relation to the nature of the revised curri- 
culum for industrial arts that were to have a significant impact on ensuing activities. First, it was 
decided that the study of industry was to. be the central objective. Second, it was concluded 
that the revised curriculum should provide an efficient and effective way to present the myriad of 
knowledges and skills utilized by industry. The first decision led the group to distill a definition 
of industry that would guide the ensuing activities. The second decision stimulated the develop- 
ment of a conceptual structure of the knowledges used in industry. More information will be 
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presented on this structure in the section that follows. 

Early in 1964, Stout received a small grant from USOE (Cooperative Research Project No. 
S*068.) This grant provided the opportunity for concentrated effort on the initial distillation of 
the conceptual structure and planning the development of the secondary school curriculum. 

In the summer of 1964, Stout received a one year grant from the Ford Foundation. This 
grant supplied the resources needed to develop the initial set of instructional materials and provide 
in-service training for ten high school industrial arts teachers. During the following academic year, 
these ten teachers tried out the materials and the curriculum in their schools. The positive res- 
sponse in this phase encouraged the Project staff to request a long-term grant from the USOE. 

In July, 1965, Stout State University started on a four year grant from the USOE (USOE 
Contract No. OE-5-85-060). This grant enabled the American Industry Project to expand to 
twenty participating teachers and seven full time university staff members. The secondary school 
curriculum development and evaluation activities financed by this grant are the basis for this part 
of the report. 



Development of the American Industry 
Conceptual Structure of the Knowledges 
Necessary to Unde rstand Industry 



Work on the American Industry structure spanned all of the phases of the Project described 
in the previous section. No specific attempt will be made to allocate individual developments to 
any one phase. Instead, an attempt will be made to capture the thinking and the procedure in- 
volved in the flow of activities employed to build the structure. 

As noted previously, an initial step in the development of American Industry was the defini- 
tion of industry. This definition delimited the field of study and the attributes in the definition 
helped to guide the development of the curriculum materials. In its final form, the definition 
read "industry is an institution in our anclatv which , intending to make a monetary profit, applies 
knowledge and utilizes natural and human resources^t o produce goods or services to meet the needs 
needs of man" , (attributes are underlined) This statement included one major change from the 
definition initially developed by the study group. After a number of discussions with people in 
industry, the attribute of profit w as added. 

During the time the original faculty group was meeting, the ideas of structure and concept- 
ual learning were coming into prominence in education. Such writers as Bruner' and Phenix were 
stressing the importance of structure to efficient and effective learning and understanding. Bruner 
was probably the best known advocate of concept learning at that point in time. The thoughts 
and writings of these two men and several psychologists provided the ingredients for the solution 
to the problem of how to assemble the numerous knowledges related to industry in an efficient 
and flexible manner. The solution was to develop ? conceptual structure that represented these 
knowledges. The working definition of concept employed by the Project was: 
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A concept is a psychological construct resulting from a variety 
of experiences (detached from the many situations giving rise to 
it), fixed by a word or other symbol having functional value to 
the individual in his thinking and behavior (Face and Flug). 

Structure was viewed as the manner in which the concepts were related. 

Using the concept as the unit of analysis, a variety of business enterprises falling with! the 
scope of the Project's definition of industry were scrutinized to determine the major concepts 
employed. Information was obtained from literature related to these businesses, from interviews 
with people functioning at various levels in representative firms, and from professors of industrial 
management and business administration. This procedure was repeated during three revisions of 
the structure. In addition, the process of developing instructional material provided another check, 
especially on the validity of the structure. The development of learning hierarchies and instruc- 
tional sequences furnished a detailed check on the relationships posited in the structure. A 
graphic summary of the result of these activities is given in Figure I. A more complete descrip- 
tion of the procedures employed to delineate the structure is presented in Appendix D. 



American Industry Secondary 
School Instructional System 



One of the major tasks financed by the last USOE grant was the development of a sec- 
ondary school program in American Industry. Successful completion of this task was contingent 
on being able to work effectively with secondary school staff members and the capability to 
produce instructional materials that were appropriate for the environment in which they would be 

used. An additional constraint was that the materials had to be valid in relation to the Project's 
conceptual structure. 

In designing the instructional materials, consideration was given to the instructional system 
utilized in the secondary schools. Major components of this system were the students, teachers, 
instructional materials, and the school environment. Once a school was selected, the nature of the 
students was to a large degree fixed. Some changes could be made in their interests but this 
had to be accomplished through the teachers and the instructional materials. Likewise, the school 
environment was relatively fixed. There may have been some money in the budget for new 
equipment and facilities but, for the most part, only minor changes were possible in this 20m- 
ponent. Thus, the only two variables open to significant variation were the teacher and the 
instructional materials. However, the teachers could not be completely retrained. A limited amount 
of time and resources were available for in-service workshops and institutes. Considering the num- 
ber "jf new content areas and instructional methods required, the time and resources were not 
adequate for optimizing the teacher variable. 

After the characteristics and limits of the other three variables were determined, the only 
remaining variable open to extensive manipulation was instructional materials. As a result, the 
decision was made to put as much effort as possible into the development of these materials and 
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Fig. I 

A CONCEPTUAL STRUCTURE 
OF THE KNOWLEDGES NECESSARY 
TO UNDERSTAND AMERICAN INDUSTRY 
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to design them to fit the existing secondary school instructional system. Thus for example, the 
instructor's guide for Level I was rewritten with greater detail. On the basis of the experiences 
with the original set of instructional materials, the first level guide was enlarged in order to pre- 
sent a more detailed suggested sequence of instruction and provide the teacher with content in the 
concept areas in which he was not familiar. Also, the media specialist designed most of the vis- 
ual media for the overhead projector since it was more accessible to the teachers than other pro- 
jection equipment. Moreover, overhead transparencies proved to be more flexible than other media. 
For instance, comments could be placed on a transparency by the instructor and he could quickly 
select one or more transparencies from his file to introduce a new topic or review one that had 
been discussed on a previous occasion. (See Appendix E for a detailed report on the media 
development.) 

This section has described the context within which the curriculum development activities 
were undertaken. Further discussion of the secondary materials will be presented in the next sec- 
tion. Additional data on the contents of the materials and their operating characteristics will be 
presented in the next section and in Chapter II. 

Developing and Revising 
Instructional Materials 



The conceptual structure of the knowledges of industry constructed by the American Indus- 
try Project provided the content and the structure for the design of three secondary school courses. 
Level I, the first course, was constructed (1) to develop a knowledge and understanding of the 
major concepts of industry and their relationships, and (2) to develop the ability to solve simple 
problems related to industry. This course was designed for typical eighth grade students. The 
major concepts and relationships studied are those on the perimeter of the core and those in the 
environment. (See Figure I.) 

Level II was designed to (1) develop an in-depth understanding of the concepts of industry 
and a refined comprehension of the relationships among thes* concepts, and (2) to expand the 
ability to recognize and solve complex problems related to industry. This course was constructed 
for ninth or tenth graf.e students. 

Level III was planned for tenth through twelfth grade students. Its general aim was to dev- 
elop the ability to apply the concepts of industry to specific problems. Problem selection would 
vary depending on the ability and interests of the student. 

These courses were designed to be sequential. A student was expected to take Level I be- 
fore he could enroll in Level II. Likewise, a student would have Levels I and II before taking 
Level III. In addition, the Projects funds for the last two years of the grant were reduced as a 
part of a general reduction in research funds at the Federal level. As a result, most of the Pro- 
ject's evaluation, experience and data were with Level I. Thus, most of the information in the 
remainder of this report will pertain to this level. 

Two of the major forces that influenced the development of the first level course were the 
Project's conceptual structure and Gagne's work on learning hierarchies (1965). As noted before, 
the structure and information associated with it provided the content and direction for sequencing 
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the materials in the course. Gagne's ideas were of significant help in establishing instructional 
sequences and the learning environments that would lead to the development of the concepts in 
the structure. 

Instructional materials for the Level I course include a two-volume instructor's guide, film 
catalog, fifty-two overhead transparencies, transparency master’s, three slide series, one film strip, 
and a set of seven student booklets. All of these materials were based upon the structure and 
most of them were processed through a develop-tryout-revise cycle at least twice. The instructor’s 
guide for Level I was revised three times in order to assure that it would function with adequate 
effectiveness in the secondary school instructional system. If the reader is interested in further 
details on the a^velopment procedures for the instructional materials, appendixes D and E should 
be reviewed. 

Level II and Level III materials were in the first develop-tryout-revise cycle at the end of 
the Project's four year USOE grant. They will need to be refined in the same manner and sub- 
mitted to the same evaluation as described for Level I. 

The develop-tryout-revise cycle mentioned in the preceding paragraphs leads to the next 
aspect of the development of the secondary school program - the evaluation system formulated to 
collect data for decision making. All three phases of the cycle were touched by the evaluation 
system. Decisions to be considered in the development of the secondary school curriculum pro- 
vided the context and questions required to guide the design of instruments and data collection. 
During the develop and tryout phases, information was collected, processed, and stored. Analysis 
of these data provided the basis for decision making in the revision phase. 



American Industry Evaluation System 



Curriculum development requires a large variety of decisions. Obviously, the developer wants 
to know if his materials enhance learning. However, it is also important to know how the mat- 
erials were used and under what conditions in order to provide a context for interpreting data. 
The major components of the evaluation system designed for the American Industry Project were 
three domains and the interactions of these domains. The domains represented the inputs, trans- 
formations, and the outputs of the system. They were entitled ingredients, processes, and pro- 
ducts respectively. The interactions symbolize the influence of one variable on another. For 
example, a student's reading ability interacts with the reading level of the instructional materials 
and influences what he learns from these materials. 

The ingredients domain encompasses all of the inputs into the learning situation. Examples 
of these inputs are the quality of the instructional materials, abilities and interests of the stud- 
ents, characteristics of the participating teachers, and the intellectual climate of the school and 
the community. 

As a course is being taught, a number of instructional processes are applied to and interact 
with these inputs. Students are exposed to instructional media, activity shetts, and booklets in 
the American Industry course. Teachers are exposed to new materials and ideas and their assoc- 
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iates become aware of a new curriculum. All of these on-going actions and activities are em- 
braced in the processes domain. This domain can generate a continuous flow of vital information 
during the time the course is in progress. However, it required systematic monitoring and a 
relatively rapid feedback process. Systematic attention to thir information can assist the developer 
and researcher in helping the teachers correctly apply the educational (experimental) treatment. 

The third domain consists of the products or outcomes of the study of American Industry, 
it is concerned with the skills, abilities, and interests of the student at the end of the course, 
opinions and impressions of the teachers, reactions by administrators and staff members at the 
school, impacts on the discipline. 

Figure 2 gives a graphic presentation of these domains and illustrates the relationships be- 
tween them. Examples oil data collection procedures and instruments for each domain are pre- 
sented in the next chapter. A more detailed description of this evaluation system and the prin- 
ciples utilized in designing it are presented in Appendix C. In addition, several of the appendixes 
contain replicas of instruments utilized in the evaluation. 

In order to have a context for interpreting the products, especially student achievement, 
control groups were established when possible. The scope and history of the American Industry 
Project did not facilitate the use of a sophisticated sampling plan that would provide a classical 
experimental research design. Both the Ford Foundation and four year USOE grants came relat- 
ively late in successive academic years. In addition, the Ford grant was not given for experimental 
purposes. At the beginning of the USOE grant there was a committement to ten teachers who 
were not selected on the basis of a sampling plan. There was also a need to expand the prog- 
ram in some of the original schools in order to make provisions for teaching Levels II and III. 

Within these restrictions, there was an opportunity to establish some experimental designs 
with equivalent control groups. In some instances, these took the form of rotational or cross- 
over designs, however, destroyed the use of the control group for follow-up purposes. Quasi- 
experimental designs were possible in two of the other schools. One of the two schools in 
which no comparison group was available served as a tryout center for evaluation instruments. 

The experimental designs in the schools varied to some degree each year depending upon 
class schedules and the students' selection of courses. However, data from previous years pro- 
vided a baseline for comparisons in some of these situations. 

Two additional concerns in the design and management of the evaluation system were the 
selection of the specific types of data to collect and the data collection schedule. To a consid- 
erable degree, the evaluation system helped to specify the data to collect and to some extent 
indicated a schedule. The type of experimental design also assisted in establishing the data 
collection schedule. However, another aid in making these decisions was the research and devel- 
opment strategy employed by the Project. Figure 3 depicts the strategy utilized in developing 
and evaluating learning expedients. The ball at the left symbolizes the basic role the Project's 
rationale played in each action and decision. Initial tryouts of materials, learning activities, or 
instructional methods usually do not provide optimrm solutions; however, they will provide feed- 
back for further refinements. These results and relationships are indicated by the spiral lines 
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Evaluation Domains in the American Industry Project 



around the solution axis and the decreasing distance between the two with each succeeding spiral. 
This strategy was applied at various times to an entire course, a unit in the course, and sub-parts 
of a unit. Its application depended on the data required in the research and the research prior- 
ities established for the Project. 

The relationships of the linear distance traversed along the solution axis during each suceed- 
mg spiral is denoted by X y'Z in Figure 3. In other words, increments of progress become 

more difficult to achieve with each successive spiral. The first trial identifies obvious areas in 
which improvements can be made. As these changes are made, smaller and more subtle problems 
come to the fore-front. 

Each spiral along the solution axis contains five distinct segments that represented specific 
steps in the development and evaluation of the American Industry curriculum. The first step was 
to state the objectives for the course or segment of instruction in as lucid terms as possible. 

Once the objectives had been stated learning expedients were selected or developed to meet them. 
At this point the learning expedients were ready for a tryout in an instructional situation. Data 
were collected prior to, during, and at the end of the tryout to provide the information for the 
evaluation. The domains noted previously identified areas in which to develop dato collection 
instruments. Interpretation of the products data was made in relation to the ingredients and 
processes data as well as in comparison with the performance of the control group. 

Upon completion of the evaluation, the research and curriculum specialist on the Project 
decided whether the objectives were met and if not, the reason(s) for not attaining them. If the 
decision was made to continue into another cycle, the same five steps were employed again. 

This process was continued until a satisfactory level of achievement was reached. "Satisfactory" 
as used in this context does not imply that the instructional materials were perfect. Rather, it 
means that in relation to the other elements of instruction beinfl developed, the one under con- 
sideration had been improved to the point where it was more productive to work with another 
one. For example, opinionnaire data from students and teachers plus achievement test data 
indicated that the student enterprise was functioning rather effectively in its initial form. (See 
Appendix H.) Thus, developmental work was stopped on the enterprise activity and attention 
was focused on other aspects of the Level I course. 

To the degree time and resources permitted, a variety of achievement measures were ad- 
ministered to the American Industry students. (See Appendixes J, K, and L for descriptions of 
several measures employed.) More than one type was used in order to isolate factual information 
from rendom error and bias factors present in the data from specific types of measurement 
techniques. For instance, objective tests allow the evaluator to sample a prescribed set of 
behaviors but they penalize the nonverbal student. Whereas essay tests allow the student to 
freely express himself, they suffer from the luck of scope in the behaviors sampled and the 
objectivity of the scoring process. Observing students in the "normal classroom setting and 
engaging them in discussion can also help to assess their competencies, but this procedure in itself 
alters the "normal" classroom situation and is open to rater bias. However, use of the data from 
all three sources in the analysis can help to identify the true outcomes. 
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To facilitate the collection of data, procedures were established for contacting the admin- 
istrators, guidance counselors, and participating teachers in the secondary schools working with the 
American Industry Project. These procedures were reproducer* and placed in the Project's Proced- 
ures Book. Each teacher received a copy to have available for reference. This book proved to 
be an effective communication and management device. 



Summary 



The American Industry Project grew from the concerns and ideas of a group of Sto»it State 
University professors. Stimulated and sustained by three grants, the Project developed a concept- 
ual structure of the knowledges used in industry and three secondary school courses based upon 
this conceptual structure. An evaluation system was instituted to provide data for the evaluation 
and refinement of these courses. The information from this system was used to modify and en- 
large the curriculum materials. Chapter II presents and discusses the data in the evaluation. 
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CHAPTER II 



ANALYSIS OF DATA AND FINDINGS 



Introduction 



The organization of this chapter is based on the three evaluation domains in the American 
Industry Evaluation System: (1) ingredients, (2) processes, and (3) products; and their inter- 

relationships. In this chapter, analyses of data related to these domains and relationships are 
described in general terms and the discussions of the various findings are presented in more detail. 
Most of the detailed descriptions of the evaluation procedures, development of the data collection 
instruments and management of the evaluation activities appear in the appendixes. The reader 
interested in specific^ details and procedures should read the appendix related to his interest A 
number of references to the appendixes are listed in this chapter to aid the reader in relating 
the two presentations. Final conclusions and recommendations appear in Chapter III. 

The ingredients domain will be presented first to provide a context for the remaining data. 
The purpose of this initial section is to give a summary view of the types of students, teachers, 
schools, and instructional materials encompassed in the American Industry Project. Summary 
statistics are presented for each of these four factors. Those for the students, teachers and schools 
are constrasted with statistics from national studies. Interactions between ingredients variables and 
processes and products variables will be described in succeeding sections. 

The processes domain will be discussed next ir sequence. Analyses of the data on student- 
teacher relationships and their interaction with the instructional materials will be reviewed. Also 
the nature of the school environment will be discussed. 

The products domain will be examined in the concluding section. Data on student achieve- 
ment and the effec s of the study of American Industry on the c tudent after graduation will be 

discussed. Also, some of the influences the study of American Industry had on student's attitudes 
and interests will be noted and contrasted with those in the control group. In addition, some of 
the impacts of the American Industry Project on Stout State University and industrial arts curri- 
culums throughout the nation will be noted. 

Data for the last three years of the project are presented in the ensuing sections. The first 
year of the USOE grant, 1965-6, was used for the development of materials. This year was also 
used to try out several of the evaluation forms. Information from these activities was very help- 
ful in planning and implementing the evaluation during the remaining three years of the Project. 
Prior to the last year of the grant the Project's budget was cut by the USOE as a part of an 

economy drive. As a result, evaluation activities carried out during the 1968-69 school year had 

to be curtailed. 
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Ingredients Domain 



Ingredients refers to all the inputs to the learning environment. These range from obvious 
factors such as the characteristics of students and the nature of the teachers to lesr salient fac- 
tors such a<> the structure upon which the American Industry instructional materials ,vere baaed. 
Each of these inputs is important in defining the factors and conditions under which the treat- 
ments were effective. 



Class Level of Students 



During the last three years of the Project, the Level I course was introduced in grades seven 
through twelve even though it was designed for grade eight. This was necessary for two reasons. 
First, a number of teachers who had worked on the original materials during the Ford Founda- 
tion grant taught at the high school level. Second, there was a need for a base of Level I alumni 
in order to provide students for the Level II and the Level III courses. Thus, although a major- 
ity of the students enrolled in the Level I course were in grade eight, students from grades nine 
through twelve and grade seven also studied the Level I course. In all schools the students were 
representative of the students who typically enrolled in industrial arts. 

A summary of the number of students enrolled by grade level in each American Industry 
course is given in Table 1. As noted above, the largest group of students enrolled in the Level 

I American Industry course were eighth graders. Only in 1966-67 did the eighth grade comprise 
less than 50% (35.8%) of the Level I American Industry students. In each of the three years the 
ninth graders were the second largest group. Thus, although older students were admitted to tho 
first level course, a large majority of the experimental students were in grades eight or nine. 

Table I gives a summary of the number of students in the American Industry classes and 
the control groups for aach of the thrae academic years between 1966 and 1969. A brief review 
of this table reveals that approximately 88% of the experimental students, those who studied 
American Industry, enrolled in the Level I course o A smaller number went on to the Level 

II and only a few selected Level III. Most of these differences can be attributed to the required 
sequence of American Industry courses, the small number of participating teachers, and the lack 
of room for elective credits in the programs of senior high school students which restricts their 
entry into industrial arts courses at this level. Also, Level III was developed during the summer 
of 1967. Thus, it was not available for use during the first two years of the Project. 

Selected Characteristics 



As previously noted, teachers and schools were not selected to conform to a specific re- 
search design; thus, it is important to review some of the characteristics of the students in the 
sample. Two factors have been selected for analysis: (1) occupation of the father, and (2) 

score on an ability test. These two variables reflect the general areas of socio-economic status 
and cognitive ability respectively. 
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TA8LE I 



Enrollment by (*rade Level in American 
Industry and Control Groups , / 96 7 - 9 



War Grade 


LEVEL 










1 


2 


3 


4 


o 


6 


TOTAL 


196667 7 


112 






62 






174 


8 


305 


.... 


.... 


231 




— 


536 


9 


202 


.... 




62 




— 


264 


10 


III 


21 




128 


— 


— 


290 


II 


62 


22 




58 




.... 


142 


12 


29 


12 


.... 


22 




.... 


63 


TOTAL 


851 


55 


0 


563 


0 


0 


1469 


1967-68 7 


47 






50 






97 


8 


472 


120 




240 


.... 


.... 


832 


9 


170 


— 




64 


— 


— 


234 


10 


124 


36 




40 


47 


— 


247 


ll 


29 


8 


.... 


12 


20 


.... 


69 


12 


25 


10 


6 


11 


10 


.... 


62 


TOTAL 


86 7 


174 


6 


417 


77 


0 


1541 


1968-69 7 














0 


8 


376 


.... 


.... 


72 


— 


.... 


448 


9 


107 


29 


.... 


49 


— 


— 


185 


10 


78 


38 


.... 


40 


18 


.... 


174 


II 


29 


1 


.... 


15 


12 


.... 


57 


12 


2 


— 


— 


.... 


7 


.... 


9 


TOTAL 


592 


68 


0 


176 


37 


0 


873 


TOTAL ALL YRS. 


2310 


297 


6 


1156 


III 


0 


3883 



i-eiW Code: I -- Level /, American Industry; 2 - Level II, American 

Industry; 3 - Level III , American Industry; 4 - Control for Level /, 

5 = Control for Level II, 6 = Control for Level III. 
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Occupations of the fathers of the students in the American Industry Project's evaluation are 
summarized in Table 2. In contrast with the SCOPE findings (Tillery, Donovan and Sherman, a, 
b, p. 14 & p. 15), there is a tendency toward slightly more farmers, simi-skilled workers in the 
evaluation sample and fewer service workers, salesmen, proprietors and managers. However, these 
differences are not marked and give no indication that the students in the American Industry 
evaluation came from an elite group. Also, all types of socio-economic classes were represented 
in the sample. 

Summaries of the ability scores for the students in the evaluation are reported in Appendix 
J. Means and standard deviations are reported for each group and for each year encompassed in 
the evaluation. Some variation in the average ability scores can be noted between schools and 
years. Average ability scores ranged from 88.7 to 118.4. However, most of the mean ability 
scores ranged between 95 and 105. A tendency for ability scores to decrease as grade level in- 
creases can be noted in the scores reported. In other words, the process of selecting industrial 
arts courses was related to ability. Within most schools the average ability levels and dispersion 
of abilities were similar for the American Industry and control groups. When equivalent control 
groups were not available quasl-experimental designs were employed. 

It is evident from the statistics reported in Appendix J that the experimental and control 
groups contained a number of students of average and less than average ability as well as those 
with high abilities. In other words, American Industry was taught to students with abilities 
representative of those found in our normal school population. 

The sample of students in the American Industry evaluation appeared to contain a cross 
section of students from our society. Students in the sample came from all socio-economic 
classes and exhibited the range of abilities typically found in public schools. 



Teachers 



One of the basic elements in the secondary school instructional system described in Chapter 
I was the teacher. In several respects, the teacher comprised the most critical variable. He con- 
trolled the learning environment and the applications of the treatment. His actions were central 
to the success <”f the American Industry secondary school curriculum. Hence, the characteristics 
of the participating teachers were especially important to the outcomes of the Project. 

All but one of the American Industry teachers were teaching industrial arts prior to their 
affiliation with the Project. The other had just graduated from college. A summary of the part- 
icipating teachers' teaching experience is presented in Tabl6 3. A wide variation of years in 
teaching were evident with a range from zero to thirty-four years. The median was seven years. 
Three teachers were starting their first year at a new school, however, the median length of time 
in the current position was 3.5 years. Seven teachers had worked in only one school system 
with the median being 1.9 jobs for the group. Two teachers had held jobs in four school sys- 
tems but they also had 13.5 and 34 years of teaching experience. As a group, the American 
Industry teachers were experienced teachers with considerable job stability. 
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Figures in the table are percentages. 



TABLE 3 



Years 



Teaching Experience of the American Industry Teachers 

TEACHING EXPERIENCE NUMBER OF POSITIONS 



Total Years 
Teaching 



Years in 
Present Job 



Number of 
Previous Jobs 



Number of 
Teachers 



0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 
24 
34 



1 

1 

1 

2 

5 

2 

2 

2 

2 



2 

2 

1 



3 

5 

1 

3 

3 

1 

1 

1 

1 



0 

1 

2 

3 

4 



7 

11 

2 



1.9 

(jobs per 
teacher ) 



Mdn ~ 7 years 



Mdn - 3.5 years 
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The participating teachers had taught the following areas in industrial arts: Woods, metals, 

drafting, printing, and general shop. Their American Industry courses were taught in these same 
laboratories. In some instances, they were able to arrange access to another lab for some of 
their activities when this seemed desirable for the teacher. 

Median age of the participating teachers was 30.2 years. The oldest was 58 and youngest 
was 24. A majority, 71% had master's degrees and several of these men had done some study 
beyond this level. Although most of the bachelor's and master's degrees were from Stout, part- 
icipating teachers had undergraduate degrees from seven other universities and master's degrees 
from three other universities. Obviously, Stout's curriculum had a large influence on the skills 
and perceptions of the participating teachers. 

Eleven of the participating teachers had formal industrial work experience. The longest 
term in industry was two years. Most of the work experience was in skilled and semi-skilled jobs 
that involved a relatively large amount of the manipulative activity. The avocational activities of 
the participants also indicated their interest in manipulative and physical activities. Hunting and 
fishing were the most popular outdoor activities with mechanics and woodworking being listed as 
favorite indoor activities. 

Since American Industry included some content usually taught in industrial arts, the teachers 
were surveyed after the 1967—8 school year to determine concept are?s in which they thought 
that they needed more information and education. The survey was conducted after the teachers 
had attended several in-service workshops and had taught at least one American Industry course. 
Hence, their responses identified the concept areas that presented the most persistent problems. 
(The complete report is reproduced in Appendix I.) In brief, more than half of the teachers re- 
sponded that they needed more information on the materials, processes, and energy concepts. In 
addition, a need for more information in each of the remaining concept areas was noted by one 
or more teachers. 

In summary, experienced industrial arts teachers were selected for the role of American In- 
dustry teacher. Considerable job stability and advanced professional training were revealed in the 
records of these men. These teachers enjoyed outdoor and mechanical activities. A survey of 
the participating teachers in 1968 indicated a need for further in-service work in several of the 
concept areas. 



Instructional Materials 



Designing the Curriculum 

Two basic elements in the design of the Level I American Industry course were (1) the 
conceptual structure of the knowledges necessary to understand American Industry and (2) the 
theory and procedures involved in the development of learning structures as described by Gagnrf 
(1965) in The Conditions of Learning. The structure identified 13 primary concepts and the 
interrelationships between these concepts that were viewed as necessary in order to understand 
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American Industry. Thus, the scope of the content was defined. Although emphasis could vary 
to some degree, the stiucture implied a balanced treatment of each concept. 

Having defined the scope and emphasis, the task of sequencing the instructional material 
remained. Since concepts are higher order cognitive processes, attention had to be given to the 
prerequisites for their attainment. Gagne's ideas related to learning hierarchies and his concepts 
concerning the conditions of learning were employed in distilling the sequence of objectives for 
the Level I course, identifying the lower order competencies required, and constructing learning 
activities. Figure 4 presents a condensed version of a learning structure for a relationship be- 
tween the concepts of communication and marketing. Similar structures were developed for the 
other concepts and interrelationships. 

The learning hierarchies suggested a definite sequence of objectives for the Level I course. 
In addition, they helped to define the requisite competencies needed to acquire the concepts in 
the structure. Since most of the concepts area:, required communication acts, this concept was 
scheduled for study early in the course. The marketing concept was deferred until it could be 
used in the students' enterprise. Once both of these concepts had been acquired, the students 
were ready to study the interrelationship (principle) between them. This was the general nature 
of the sequence of the objectives in the Level I instructors guide. 



To check the sequence of the objectives in the completed guide against the original design, 
a content analysis procedure was developed. In brief, this procedure involved the review of each 
lesson and the recording of the concept areas and interrelationships studied. (See Appendix F 
for a detailed description of the content analysis.) These data were then punched on tab cards 
and processed. Analysis of the results revealed that the sequence and content identified in the 
original plan for the Level I guide were attained. Teacher feedback contained eight ratings, 

1.5% of all ratings, indicating that lessons in this guide were out of sequence. One-half of these 
responses were related to the sequence of the four lessons in Unit 1. (See Appendix F for a 
detailed analysis of the teacher's feedback.) Later study indicated that the last two lessons in 
Unit I could be interchanged without any negative effects. Thus, with an optional sequence for 
the lessons in Unit I, the content sequence in the Level I Instructor's Guide appeared to be 
appropriate for the teachers in the Project's sample as well as being psychologically sound. 

In review, Gagnds procedures for structuring content were used to define the sequence of 
objectives and the hierarchy of competencies in each concept area. Content analysis of the in- 
structor s guide indicated that the lessons in the guide followed the sequence planned. Teacher 
feedback revealed that the sequence was appropriate. Therefore, it would appear that the Pro- 
ject's structure and Gagnrfs procedures were valid. (Richard Gebhart's paper reproduced in Ap- 
pendix D presents a more complete description of the procedures utilized to design the secon- 
dary school courses.) 



Instructors Guide 



The instructor s guide had several functions. The order of the lessons established a sequence 
for the course and each lesson integrated the required learning resources and activities for the in- 
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Fig. 4 Learning Structure for a Principle Involving 
Communication and Marketing 
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structor. Also, most lessons contained detailed content. This detail varied with the most exten- 
sive presentations occurring in the concept areas most foreign to the participating teachers, for 
example, the lesson on market analysis contained a detailed description of sampling procedures 
and methods for interpreting the data that resulted from the survey. 

The final version of the Level I Instructor's Guide was based on the feedback from the 
participating teachers. It first evolved from resource units related to each of the concept areas 
to a guide which identified a sequence of learning experiences and was correlated with the orig- 
inal resource units. Then, when the participating teachers felt that this revision did not provide 
enough detail and direction, the content in the resource units was integrated with the lessons in 
the guide to provide one detailed reference for the teacher. In addition, some sections were 
amplified to solve problems identified in the teachers' feedback. During the third year of the 
Project, the detailed guide was used by all instructors teaching the Level I course. At the end 
of this year they responded as indicated to the statement: "This year's guide is much better 

than last year's." The detailed 

SD D U A SA NR 

0 0 3 7 9 2 

version of the guide appeared to satisfy several of the teachers' needs in the areas of content and 
sequence of learning activities. 

The first drafts of the Level II and Level III guides were written during the USOE grant. 
However, there were not enough time and resources to complete the field testing and to revise 
them. Preliminary content analysis revealed a need to modify the content, sequence, and empha- 
sis in the Level II guide. Similarly, the Level III guide needed to be revised. Both would pro- 
fit from the evaluation procedures utilized for Level I. Moreover, the first level guide could also 
be improved through continued evaluation. 



Media 

As lessons were developed for the instructor's guides, the media specialist reviewed them and 
developed media correlated to specific segments of the lessons. Les.on writers also suggested media 
for their lessons. As a result, a close tie between the lessons and the media was achieved. After 
using the media, on'y 9% of the participating teachers responses indicated that the instructional 
media being rated were not correlated with a given lesson. (See Appendix I for the complete 
study.) 

Two major principles were used in developing the media for Level I. First, the media 
specialist set out to make the media functional. To achieve this, most of the visuals were done 
on overhead transparencies to allow visual input by the teacher during projection and to accomo- 
date the projection equipment available in a majority of the participating schools. Also, printed 
narrations ware included with the filmstrip and the slide series to reduce the need for tape 
recorders which were scarce in most of the oarticipating schools. Second, color and cartoon 
characters were employed to make the media interesting. (John Zuerlein has described and ill us- 



2.21 




Ah 



trated these principles in a paper which is reproduced in Appendix E.) 



Feedback comments from the participating teachers revealed that the media were an essen- 
tial part of the instructional materials package. 



Student Booklets 



Seven student booklets, one for each unit of instruction were written for the Level I course. 
Booklet One introduced the student to industry. A brief history of industry was presented in 
booklet Two. Booklets Three, Four, and Five were based upon the story of the development and 
operation of a new enterprise. This story was part factual and in part fictional and provided 
referents for all of the major concepts in the Projects structure of the knowledges utilized in 
industry. The themes for Booklets Three, Four, and Five were "Organizing an Industry", "Oper- 
ating an Enterprise", and "Distributing Products and Services", respectively. In booklet Six, the 
student was exposed to some ideas on the future of industry. Booklet Seven provided informa- 
tion to assist the students in starting their own enterprise in class. The purpose of this book was 
to aid the students in acquiring a concrete experience which would integrate and reinforce me 
ideas and concepts included in the prior booklets. 

Feedback from the teachers indicated that the referents provided in booklets Three, Four, 
and Five were not perceived by all of the students. In response to the statement "most students 
could identify the concepts used by Wes as he managed Pronto Prefabs," 9.5% disagreed, 33% 
were undecided, and 47.6% agreed. However, as will be noted in the processes section, the 
booklets were not always used as intended. Thus, in these ratings the functional quality of the 
bookie's may oe confounded with the manner in which they were used. 

The booklets were designed and written for eighth grade students. An analysis of the 
reading level with the Dale-Chall formula produced the following data. 



Booklet 


Title 


Reading Level 


1 


Industry Today 


7th - 8th Grade 


2 


Evolution of Industry 


7th - 8th Grade 


3 


Organizing an Industry 


8th - 9th Grade 


4 


Operating an Industry 


9th • 10th Grade 


5 


Distributing Products and Services 


10th • 11th Grade 


6 


The Future of Industry 


9th - 10th Grade 


7 


Student Industry 


9th - 10th Grade 



The reading levels in the first two booklets met the criterion established. Booklet Three 
was at the upper limit of the criterion. Booklets Four, Six, Seven were slightly above the 
criterion and booklet Five was distinctly above the limit established. The major cause of the 
higher reading levels in booklets four through seven was the number of words not on the Dale 
list. Few technical and business terms are included on the Dale list. Thus, the introduction 
of these terms in booklets Four Through Seven increased their reading lev *1. However, it 
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was concluded that proper terminology should be used when possible. Hence, the curriculum 
specialist decided to use the booklets in the participating schools. A report of the student's 
opinions of the reading level will be given in the processes section. 



Participating Schools 



As noted in Chapter I, the American Industry Project did not have an opportunity to 
use a sampling plan in selecting its participating schools for this study. Most of the secondary 
scho iis and teachers involved in the original development of the American Industry Project 
financed by the Ford Foundation wanted to continue with the Pnject. After the four year 
grant was received from the 'JSOE, a few additional schools were added on the basis of an 
interest expressed by one or more of the teachers in their industrial arts department and the 
potential for implementing all three levels of American Industry. This resulted in a set of 
participating schools located in small to medium sized cities. In 1966, the Protat sought and 
acquired a working relationship with a large city. Ensuring data and comparisons are pre- 
sented to define the characteristics of the participating schools and contrast these traits with 
statistics for a national sample of schools. 

Three of the most important school characteristics associated with student perormance 
that were identified in the studies of the American High School by Dailey (Flanagan, Dailey, 
Shaycoft, Orr & Goldberg, 1962 pp. 9-36), were (1) beginning teacher's salaries, (2) amount 
of teaching experience of staff, and (3) per pupil expenditures. These factors had the 
highest and most exclusive relationship with student performance. Table 4 presents a summary 
of the data from Project TALENT and the American Industry participating schools. Char- 
acteristics 12, 7, and 10 in Table 4 contain the information on the variables identified above. 



The average salary in the 1965-66 school year for a beginning teacher in the participating 
schools was $5150. In comparison with NEA statistics (1966), the average for the participating 
schools was slightly lower. However, the range of starting salaries in the sample of participating 
’schools was typical of those offered in the region. The difference between the mean for the 
American Industry schools and the Project TALENT sample could be made up by average 
annuii raises of $280. Thus, there does not appear to *»c any significant difference between 
the. two samples in the starting salaries of teachers. 

Turning to the experience variable, the staffs of the participating schools have slightly 
more experience than the national sample with a higher proportion of teachers having twelve 
or more years of teaching experience. However, there is little difference in the lower range 
of experience. Both samples have approximately 30% with 0 to 8 years of experience. Most 
•>f the differences come in the 9-11, 12-14, and 15 and over categories. Since the main 
variation is at the higher experience level, there would appear to be little difference in 
quality on this variable. 

Average per pupil expenditure in the American Industry schools, during the 1965-66 
school year was $538.23. This contrasts with an average of $533.00 reported in the NEA 
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This is average starting salary reported in the NEA Research Bulletin for the 150 largest school *i 
in the ini ted States. In 1060 the mean was $4145 for this same sample. 



Research Bulletin (1966a, p. 39). Thus there was no difference on this variable. 

In terms of the three variables just discussed, there is no meaningful difference between 
the American Industry schools and schools in general. On the basis of Project iALENTs 
findings (1962), the educational environment in the participating schools was representative of 
the environment in schools in general. 

Table 4 also presents a number of additional characteristics that help in defining the 
nature of the participating schools. All of the participating schools were public institutions 
with state or regional accreditation. Project TALENT reported a slightly lower total for 
their national sample. A large majority of the participating schools had previOw*«ly participated 
in the tryout of an experimental curriculum (90^ and had tried their own special curriculums 
(80%). Experiences with experimental curriculums was considerably lower in the Project 
TALENT sample. However, five years had intervened between the two surveys, during which 
time a number of curriculum projects had been functioning. Thus, this difference may re- 
flect more of a change in curriculum development activities than a difference in samples. 

Absenteeism rates and the amount of homework expected were very similar for both 
groups. The average size of the senior class varied drastically between the two samples. How- 
ever, NEA 0966a, p. 37) reported that the number of high school graduates had increased 
45.2% from I960 to 1965. Increasing project TALENTS's sample average by 45.2% gives a 
mean of approximately 124 students per senior class. This is still considerably lower that the 
mean for the participating schools. Considering this evidence in concert with the population 
size of the communities in which the participating schools were located median ■ 10,000 and 
a range from 3,500 to 750,000 it is evident that the sample of American Industry schools 
represented larger schools and communities than those found in the national sample. 

Principals in the participating schools estimated that 41% of their seniors were enrolling 
in a four year college after graduating from high school. In the NEA report, it was estimated 
that 39.1% of the high school graduates in 1965 entered a four year college (1966, p.37). 
Thus, the college attendance rate in the American Industry schools was similar to that for 
all high schools in the nation. 



One measure of the socio-economic status of students is the type of residence in which 
they live. A comparison of quality of housing in each group revealed that the students in 
the American Industry sample came from more expensive homes. In contrast with the 
national sample in which 53.6% of the students were estimated to live in modestly priced 
homes, 90% of the students in the participating schools were said to be living in modestly 
priced homes. While schools in the national sample classified 23.2% of their students as 
living in predominatly low cost homes, only 5% of the students in the participating schools 
were classified in this manner. None of the participating schools was identified as being, 
in a low income area; however, 10.8% of the schools »n the national sample were. (Two 
special associations which the American Industry Project developed with disadvantaged schools 
will be discribed at the conclusion of this chapter). From the quality of housing measure 
it would appear that the status of the American Industry sample was slightly better than the 
average socio-economic level in the Project TALENT sample. However, when the occupational 
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level of the students' fathers (summarized in Table 2) was considered, there would appear to be 
little or no difference in the socio-economic status between the two groups. 



3n summary, with the exception of size, the participating schools were very similar to the 
schools in Project TALENT'S national sample. Participating schools were located in more medium 
sized cities than were encompassed in the national sample. 

Summary 

The American Industry conceptual structure of the knowledges of industry and Gagne's 
concept of learning hierarchies were used to design the structure and select the contents for 
the American Industry secondary school curriculum. Instructional materials for the first level 
course were field tested and revised three times. Content analysis and teacher feedback in- 
dicated that the Level I Guide produced in the summer of 1967 had an appropriate sequence 
and valid content. 



Students in the evaluation exhibited characteristics similar to those in a national sample 
identified by Project TALENT. The students' abilities and their father's occupations were very 
similar to those found in the Project TALENT sample. School characteristics also paralleled 
those in the national sample. Thus, although the American Industry sample was not selected on 
the basis of a formal sampling plan, the schools, teachers, and students possessed characteristics 
similar to those in the Project TALENT sample. As a result, it would appear that American 

Industry has been taught under conditions and to students typical of those found in most regions 
of the United States. 



Processes Domain 



Data in this domain are directly related to the activities initiated to direct, sustain, and 
enhance learning. In this domain there was concern with the interactions between the teacher and 
student, teacher and instructional materials, and student and instructional materials. In addition, 
the interactions between these components and the school environment were of interest. Infor- 
mation on diese variables provided an indication of the operating characteristics of the instructional 
materials and the nature of the American Industry classes conducted in the participating schools. 
The ensuing sections present information gathered on each of these interactions. Unless other- 
wise noted, the data reported are for the final version of the Level I Instructor's Guide and the 
Level I course defined by this guide. 

Student - Teacher Interaction 



In a study carried out during the first year of the USOE grant, Egan (1966) concluded 
that students gave evidence of a more positive attitude for this American Industry class than 
for the remaining classes in their program of studies. Data for this study were collected in 
the following manner. Students in the American Industry classes were asked to list in rank or- 
der the classes they liked and disliked. They also were requested to list a reason for their rank- 
ing. 
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The responses of the students were summarized and the preferences for various courses were 
analyzed. Egan (p. 33) found that 60% of the students reported that they liked their American 
Industry class and 21% indicated that they disliked their American Industry class. In contrast, 
approximately 45% of the students responded that they liked English, math, and science; and 
47% reported a dislike for English, while 27% indicated a dislike for math and science. The 
chi square value for these differences was significant at the .01 level. 

Egan also asked the students to list their specific likes and dislikes in relation to the 
American Industry course. After reviewing the responses on this section of his instrument, Egan 
(p. 41) concluded that the teacher's managerial ability, the utility of the course, and the type of 
laboratory activities included in the course were the three most important causes of likes and 
dislikes. 

Egan's study was replicated during the 1967-8 school year and very similar results were 
obtained. At the end of the 1967-8 school year 56.5% of the students included American In- 
dustry in their “like" list and 22.3% stated that they disliked it. Similarly, the teachers' manag- 
erial ability, the nature of the lab activities, and the relevance of the course remained as the 
prime factors in liking and disliking the course. (See Appendix 0 for a detailed report of 
these studies.) 

On the basis of Egan's study and the results of the later study, the Project encouraged the 
development of relevant learning activities and simulations of industry. In addition, the participa- 
ting teachers were encouraged to use appropriate teaching techniques with the diversity of schools, 
teachers, and students encompassed in the Project's evaluation. It was not possible to completely 
satisfy this need. However, the opinions of the participating students indicated that they were 
generally satisfied with American Industry and with many of the specific activities to which they 
were exposed. 

Teacher - Instructional Materials Interaction 

The evaluative comments of the participating teachers indicated that the Level I course mat- 
erials and instructor's guide functioned effectively in their American Industry classes. Each teacher 
who had used the final version of the Level I guide was asked to rate a series of statements re- 
lated to its qualities. All of these statements were rated favorably with two exceptions - (1) 

90% had difficulty completing the course on time and (2) most wanted more flexibility in the 
guide. 



Favorable comments were received on the scope, format and quality of the instructional 
materials. Only 10% of those responding (N=20) had any problem understanding any part of the 
instructor's guide. A majority (60%) thought that the guide was essential to teaching the course. 
And, 90% responded that the format of the guide was easy to follow. Three teachers (15%) 
thought that the lessons slviuld have had more content. None of the participating teachers 
thought that the format of the lessons was not functional. 

The instructional media were also evaluated favorably. None of the participating teachers 
thought that straight lectures held the student's attention better than the combination of overhead 
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transparencies and their lectures or discussions. In addition, approximately 75% of the teachers 
concluded that the instructional media were an important part of the materials supplied by the 
Project. (Appendix I contains a complete summary of the teachers' responses on the 1967 8 
End of Year Report.) 

Participating teachers also rated each of the lessons they taught. A rating form was included 
at tne end of each lesson and the teachers were asked to evaluate each lesson as soon as possible 
after it had been completed. The second item on this evaluation form asked the teacher to rate 
the overall quality of the lesson as "excellent", "good", "fair", or "poor". Numerical values of 
1, 2, 3, and 4 were assigned to these categories respectively. (See Appendix F for a detailed 
presentation on ihe feedback system and a complete summary of the feedback.) 

Analysis of the teachers' responses revealed that the average rating given all lessons was 
1.79 or somewhat better than "good". Only 10 of approximately 600 ratings fell in the "poor" 
category. Many of the low ratings were assigned to the lessons in Unit II - Evolution of Industry. 

This unit dealt with the history of industry and proved to be a difficult unit to develop and 
teach. 



Ratings for the lesson objectives and content outlines were favorable. Less than 4% of the 
responses indicated that the objectives were not clear. Approximately 9% of the ratings on the 
content outline assessed it as lacking detail or not being logical. However, the only systematic 
pattern of negative ratings was found in the responses to Unit !!. 

Ratings and comments made by the participating teachers suggested the need for more act- 
ivities to develop the concepts in the structure. This was primarily a problem related to the 
quantity of a< ties since the response to those listed in the lessons was favorable. Most of the 
teachers developed their own activities to supplement those in the guide. On the basis of this 

feedback, it would appear that there is a definite need for the design of additional activities that 
facilitate concept learning. 

Unit rating forms were included at the end of each unit. Responses on these forms parall- 
eled those on the lesson ratings. The average unit rating was \.83 in contrast to 1.79 for the 
lessons. No "poor" ratings were given to any unit. Again, Unit II had the lowest evaluation. 

Provisions were made in the design of the Level I Instructor's Guide to gather data on 
three additional questions of interest to the staff members writing the guide. First, how much 
detail should the content outline in the lesson contain? Second, do deductive and inductive 
lessons meet with the same level of approval? Third, do the participating teachers prefer lessons 
which involve making products more than those which stress cognitive learning? 

To obtain information on the first question, detailed and outlined lessons were included in 
each of several concept areas. An attempt was made to have each type cover material of equal 
difficulty. Analysis of the ratings of these lessons disclosed significantly more "poor" ratings 
for the outlined lessons. (See Appendix F for the data and analyses related to these questions.) 
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A similar procedure was employed to acquire data on the second question. Deductive and 
inductive lessons were written in each of several concept areas. Analysis of the ratings revealed 
no significant differences. It appeared that both types were equally well received. 

To answer the third question, ratings for lessons dealing only with production of a product 
were contrasted with those concerned with the development of concepts and knowledges related 
to industry. A significant difference in favor of the production lessons was observed in the 
ratings. 

The lesson rating form also asked the teacher to assess the contribution of the student 
booklets to the attainment of the lesson objectives. Approximately 12% of the responses indi- 
cated that the booklets had little relevance to the lesson. Often this rating was amplified by the 
comment "the lesson should be based on the booklet". Statements of this type suggested that 
the booklets were not always used as intended. In addition, comments in the students' evalua- 
tion of the course indicated that the booklets were not always utilized in a manner that would 
help establish and reinforce the concepts included in the lessons. 

When specific readings in a student booklet were identified in a lesson 97% of the ratings 
gave evidence that they had been used. However, when pages were not identified the teachers 
tended not to use the booklets. A similar problem was observed with the use of additional ref- 
erences such as periodicals and pamphlets. About 8% of the references of this type listed in the 
lessons were utilized. As a result, of this finding, the Project procured and distributed a number 
of pamphlets and issues of periodicals germane to the courses. 

The favorable ratings found in the teacher feedback were substantiated by the observations 
of the Project's supervisor of participating teachers. Each participating teacher was visited several 
times during the school year. Analysis of the supervisor's recorded observations revealed that 
50% of the teachers followed the guide and made few if any modifications. Another 45% 
made some modifications but retained the basic structure of the guide. One teacher developed 
his own course. 

The importance of the guide can be observed in the test results Trom the school in which 
the instructor developed his own course of study. Means for the Achievement Test for the 

1966- 7, 1967—8, and 1968—9 school years are plotted in Figure 5. During the 1966—7 
school year, this teacher followed the general intent of the Project's instructor's guide. In the 

1967— 8 school year he developed his own course. At the end of the 1967-8 school year this 
teacher moved to another school system. A new teacher was oriented to the instructional mater- 
ials during the summer. Since this was his first experience with American industry, he followed 
the instructor's quide. Comparing these means with a one-way anaylsis of variance produced an 
F value of 8.73 with 2 and 243 degress of freedom. This value is significant at the .01 level. 

All of the boys at the grade level were enrolled each year. The average I.Q. for the three 
years were 107.7, 106.7 and 106.1 respectively. Thus, it would appear that the major variable in- 
fluencing performance was the degree to which the intent of the Level I Instructor's Guide was 
followed. 

To summarize, the final version of the Level I instructional materials, first used during 
1967-8 school year, were favorably evaluated by the participating teachers The instructional 
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Performance 



W iv teacher used guide) 




Fig. 5 Effects of the Use/Non-Vse of the 
Instructor's Guide on Mean Performance Score 
on the American Industry Achievement Test 



Source SS d.f m.s. F 

Years 1189.2? 2 594.64 8.73 * 

Within years 16552.75 243 68.12 

Total 1742.02 245 

* P>.01 



2.31 




56 



media were judged to be very effective and essential to teaching the course. Student booklets 
contributed to the instructional system when their use was specified in the instructor’s guide. 
The instructor's guide proved to be a critical part of the instructional system. 

Student - Materials Interaction 



As mentioned in previous sections, student booklets for Level I were written for eighth 
grade students. Most of the lessons in the instructor's guide listed readings in these booklets. 

After a booklet had been read, the American Industry students were asked to evaluate its char- 
acteristics by completing a rating scale especially designed for this purpose. A Likert format was 
used in this evaluation form. (The items and a summary of responses by grade level and year 
are given in Appendix G.) 

Student ratings of the booklets were collected during the first two years the booklets were 
used. At the end of the second year, it was decided, for reasons of time, finances, and printing 
deadlines, that the booklets would not be revised during the last year of the Project. Since data 
for the 1966-67 years were available on these booklets, additional ratings were not requested. 

Tables 5 and 6 present summaries of the responses to all booklets. The Likert statements 
used in the rating scale are listed in each table. Four items, 1, 9, 10 and 12, give a direct 
indication of the quality of the student booklets. Item 1 in the rating scale asked the responders 
to mark whether they agreed or disagreed that the booklet was interesting. In 1966-7, 59.2% 
thought that the booklets were interesting and 21.9% responded that they were not. Very similar 
results, 58.8% and 20.7% were found during the 1967-8 school year. 

Students' perceptions of the amount they had learned from reading the booklets were re- 
vealed in the responses to items 9 and 10. During 1966*7, 58.7% of the students responding 
indicated that they had "learned a lot" from the booklet and 24.7% said that they had not 
achieved this level. The following year the students were more undecided with aO.1% agreeing 
that they had learned a lot and 21.7% disagreeing. Since this was a fairly strong statement, it 
would appear on the basis of students' perceptions that the booklets functioned effectively as 
far as learning was concerned. 

Item 10 complemented item 9 in that it asked the student to indicate his prior acquaintance 
with the contents of each booklet. In each year, approximately 21% thought they knew most of 
the contents of a booklet before they read it. This figure is almost identical to the percentage 
who felt that they did not learn much from the booklets and who indicated that the booklets 
were not interesting. 

During both school years, approximately 76% of the responses to item 12 concurred that 
the ideas in the booklets were worth knowing. Only slightly more than 8% disagreed with this 
statement. Obviously, t f : students thought that the material in the booklets was worth knowing. 

In summary, the booklets were favorably received by the majority of the students. However, 
as will be discussed in the ensuing paragraphs, improvements are needed in some areas. 
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As mentioned previously, the ratings refer to the first and only edition of the student book- 
lets. Certain flaws have been indicated by the students' responses on the rating scale (NRS) and 

the participating teachers' qpmments. One basic problem centered on the names given to the 

, / • 

characters in the booklets. The writer used the first names of the Project staff members which 
resulted in the two hoys in the stoty being named Dick and Dan. These names caused some 
students to associate the booklets with their elementary school texts. Responses to item 7 re- 
vealed that about 36% of the readers felt that the booklets were written for someone younger. 

The simple act of changing names might reduce this unfavorable set on the part of some of the 
students. 

Another weakness of the booklets was the limited use of illustrations. The Project's printing 
process limited the type of graphic illustrations that could be included. This limitation was re- 
flected in the students' responses to item 16. In 1966-7, 42.6% thought that the pictures helped 
to understand the booklet. During 1967-68, this dropped to 36.6%. It would appear that the 
booklets could be measurably improved with better illustrations. Making these two revisions 
should improve the students' interest in the reading materials. 

Trends in ratings by grade level of the responder may be reviewed in Appendix G. 

Interactions with Environment 



Learning activities in the American Industry courses were not designed to take place only 
within the confines of the classroom or laboratory. The student enterprise activity required cust- 
omers from the school and community. Moreover, numerous referents for the American Industry 
concepts existed in each of the communities where the participating schools were located. Failure 
to make use of these resources would have been a serious omission. 

Since American Industry was a curriculum innovation and the courses were in a developmental 
stage, the opinions, cooperation, and decisions of the administrators in the participating schools 
were very important. Also, the American Industry Project was interested in the reactions of the 
participating teachers' colleagues to the new course. Each of these factors would have some 
influence on the success of the American Industry program in the participating schools. Thus, 
provision was made in the Project's set of evaluation instruments tc collect information on these 
variables. 

During the 1967-8 school year, approximately 40% of the participating teachers had visiting 
speakers in their American Industry classes. About 50% of the teachers scheduled a field trip for 
their American Industry students. Although specific data w^re not collected on these learning 
activities for the other three years of the Project, reports and comments made by the participating 
teachers indicated that the 1967-8 data were representative of the remaining years. 

Student interest in field trips warranted greater use of this learning expedient by teachers. 
More than 82% of the 1967-8 American Industry students responded that they would like to 
visit a business to see how it operates. For those that did visit an industry, the effect was 
lasting. Several students commented in the follow-up study that the field trips were very 
effective learning experiences. 



With one exception, the participating teachers used the complete enterprise activity in their 
schools. In one school, administrative regulations prohibited the sale of products. To accommo- 
date this situation, the teacher modified the enterprise to produce gifts for needy children and 
contracted with the school to manufacture articles required by the staff. Of those who used the 
complete enterprise, only two were uncertain if it created any problems in their school. The 
others responded that it did not cause any negative effects. 

A large majority (71.4%) of the participating teachers thought that theii administrators 
considered American Industry courses an important part of their school's curriculum. Only one 
teacher (4.7%) responded that his administrators did not think this course was important. More 
uncertainty was evident in the responses to the statemert "If there were other teachers of Amer- 
ican Industry in my school system, I believe the administration would initiate all three levels of 
American Industry in the school curriculum." Approximately 38% agreed with this statement, 
another 38% were uncertain, and 19% disagreed. 

The opportunity for expansion did not appear to be as good as the evaluation of functioning 
American Industry courses would seem to indicate. However, the favorable response to the init- 
ial American Industry courses should jro*'de a basis for persuading administrators to expand the 
American Industry program. Considering the newness of the American Industry courses and their 
significant departure from traditional industrial arts courses, it would appear that American Indust- 
ry had made a favorable impression on the administrators in the participating schools. 

Attitudes of the participating teachers' industrial arts colleagues seemed mixed Approximately 
33% of the participating teachers thought that their colleagues had a favorable attitude. Another 
33% were uncertain. About 28% thought their associates had a negative attitude. 

More interest seemed to be expressed by teachers from other school systems. About 43% 
of the teachers reported that they had been requested to make one or more presentations. And, 
80% said that they had had visitors in tl.jir American Industry classrooms. 

To briefly summarize, extensive use was made of field trips and visiting speakers. Students' 
interest and evaluative comments indicated that more field trips could be taken. Enterprise act- 
ivities, especially the sales activities, did not appear to cause any problems for most teachers. 
However, some school systems had special regulations that needed to be identified prior to 
initiating this activity. 

According to the teachers' reports, their administrators had a favorable attitude toward the 
American Industry courses in their schools. Attitudes of colleagues in the industrial arts depart- 
ment were more mixed. However, this was to be expected since American Industry was a 
challenge and somewhat of a menace to these men. The participating teachers noted considerable 
interest from visitors and a number of requests for presentations. Overall, the interactions be- 
tween the American Industry instructional system and its environment were positive. 

Student - Teacher - Instructional Materials Interaction 



Observations made by the supervisor of participating teachers during the 1965-66 school 
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years indicated that the teachers needed a more detailed instructor's guide. During this year, 
a number of participating teachers reported problems in planning their instructional sequences. 



On the basis of this feedback, a more detailed guide was written for the 1966-67 school 
year. This guide was in turn revised for the 1967-68 school year. Reports of supervisory visits 
at the participating schools for the laitar year gave evidence that the teachers, students, and in- 
structional materials had formed an effective instructional system in a majority of the participat- 
ing centers. A summary of the supervisor s observations revealed that 7% of the students were 
enthusiastic about the course and 68% were interested. The remaining 25% were identified as 
"cooperative". None of the classes were rated as "indifferent", "uncooperative", or "rebellious". 

No doubt the presence of a visitor stimulated the students to behave somewhat better 
than normal. However, 50% of the teachers commented to the supervisor that their students 
were interested in the course. Another 30% said that the course was going fine or that the 
instructional material were good. Since these comments were unsolicited, there appeared to be 
a positive interaction between the students, teachers, and instructional materials in most schools. 

Enterprise. 

A review of the student evaluation of the enterprise activity gives an insight into the quality 
of one of the major learning activities in the Level I American Industry course. This activity 
was also of special importance because it embraced all of the concept areas in the structure. 
Hence, student ratings should provide an indication of their interest in the course content. 

A survey of student opinions taken aCter completion of the enterprise used in Unit I of the 
Level I course revealed that 83.5% of the participating students like the student enterprise. Only 
6.3% reported that they cLsIiked it. (See Appendix H for the detailed report of this study.) 
Nearly 77% responded that they liked the group wcrk in the enterprise and almost 75% of the 
students said they liked the course at this point. 



Less than 12% cf the students thought selling the products produced in the student enter- 
prise was not interesting. Most students (75%) said that making a profit added interest to the 
enterprise. Slightly more than 77% of the students thought that they learned a lot about industry 
during the first unit. 



Observations made by the supervisor of participating teachers and the students' reactions to 
the enterprise activity indicated that the Level I courses were functioning effectively in most 
schools. Students reported a high level of interest in the student enterprise activity. 

Summary 



The processes section has presented information on the operating characteristics of the 
American Industry instructional system. Teacher, student, and teachers perceptions of administ- 
rator reactions to American Industry were found to be positive. Data gathered during the eval- 
uation indicated that the instructional materials were not always used as intended. When these 
modifications were counter to the objectives and principles utilized in designing the materials this 
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fact was reflected in the opinions and performance of the participating students. 

Review of the participating students' ratings of the student booklets and the student enter* 
prise activity indicated they thought that the knowledges encompassed in the American Industry 
courses were valuable. They also thought that the student enterprise activity was interesting 
and a worthwhile learning experience. 

The effectiveness of including evaluation as an integral part of the curriculum development 
process can be observed in the refinement of the instructional materials used by the Project. 

Both the instructor's guides and the instructional media were enhanced through the use of 
evaluation feedback from the participating schools. The instructor's guide for Level I was sign* 
ificantly improved and the experience with this guide provided a basis for a model for the guides 
needed for Levels II and III. 



Products 



T^e central focus of the evaluation was to assess the effects of the study of American 
Industry. Of prime interest were answers to the following questions: 

1. Do students who have studied American Industry have a more detailed knowledge 
of the concepts and structure of knowledges used in industry than those who have 
not? 

2. Do students who have studied American Industry have a different perception of 
industry than those who have not? 

3. Does the study of American Industry have any impact on career Voices and decisions? 

4. Does the study of American Industry have any influence on the students' ability to 
think conceptually? 

Each of these questions will be considered in the following sections. 

Effect of the Study of American Industry on Concepts and Structures 

One of the major goals of the study of American Industry was to develop a conceptual 
knowledge of industty. As noted previously, considerable effort was given to tasks of distilling 
a conceptual stricture of these knowledges and translating this structure into instructional mater- 
ials. The previous section in this chapter, processes, reported the results of the evaluation of 
the instructional materials and procedures. 



Achievement Test. An achievement test was designed to measure the cognitive outcomes 
of the study of American Industry. In order to facilitate data gathering, multiple choice items 
were used in this instrument. Since concepts permit rational responses to unique stimuli, problem 
situations were structured for the stems of the multiple choice items. Standard procedures for 
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administering the achievement test were established to avoid disseminating the contents of the 
test items to the teachers and students. A detailed description of the procedures employed in 
writing the achievement test are given in Appendix J. Results of the comparisons between the 
American Industry groups and control groups within several school settings are also given in this 
appendix. 

A ba r ic concern in the development of the achievement test was content validity. Through- 
out the development sequence the contents of the items were checked against the project's 
conceptual structure to assure a valid instrument. A second concern was the accuracy of the 
scores on the achievement test. The median reliabilities for grades eight, nine and ten through 
twelve were .81, .84, and .85 respectively. These values were considered adequate for the Pro- 
ject's evaluation. 

Table 7 presents a summary of the comparisons made between the Level I American Ind- 
ustry groups and control groups on the achievement test data. Statistically significant differances 
in performance were found in 33% of the comparisons. In each instance, these differences fav- 
ored the American Industry groups. Another 11% of the comparisons resulted in F values that 
app'aached significance (.10>P>.05). Again, each of these comparisons favored the American 
Industry class involved. In only one experimental setting did mean performance on the achieve- 
ment test for the control group exceed the mean for the American Industry group. The latter 
finding also indicated that the American Industry courses had some effects since, if there were 
no effect, the control group means should exceed the experimental group means as frequently as 
the experimental group means exceed the control groups meanr. Very similar results were found 
at Level II. 

Table 7 also presents information on the effects of the length of course, semester vs. year 
and the degree to which the instructor's guide was followed. Programs one year in length pro- 
duced more significant differences than those which were of semester length. Approximately 
41% of the year long programs produced significant differences. In contrast, 30% of the semester 
courses resulted in significant differences. 

None of the American Industry classes in which the instructor's guide was not followed 
performed significantly better than its control group. This reflected both on the value of the 
guide and the fart that non-use of the guide typically resulted >om a difference in ohilosophy 
between the teacher involved and the Project. Since the achievement test was based upon the 
Project's concepts and structure, programs which moved away from this rationale had less chance 
to produce significant differences in performance. When only the programs taught by the 
teachers who followed the guide are considered, approximately one-half of the comparisons are, 
or approach being significantly different. 

During 1967—68, the impacts of major revisions in the instructional materials can be ob- 
served. For this year the instructor's guide was completely revised and new media were developed. 
The long range results, however, may be seen in the improved performance in 1968—69. 

Although the results of the comparisons may not be as dramatic as one might hope for in 
the outcome of a new curriculum, two points must be kept in mind. First, '968—69 represented 
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significant ' *, no control group available 



the initial year in which all of the Level I materials were utilized l> »ho had prior 

experience in their use. Second, the achrsver v ent test m<! js designed to , .* ‘U# data for the im- 

provement of the American Industry curriculum. TKs, most of the items in the test presented 
novel situations in which the students had to apply the knowl* and concepts they had ac- 
quired. For example, in the following -test item, the distracto's disclose whether the student had 
information on the costs, insulative qualifies, and strength of common materials. If, for example, 
a student thought that materials were selected for their strength, he would select O as the appro- 
priate choice. 



Why would mahogany paneling be used in the 
office of the president of an enterprise? 

A. To reduce building cost 

* B. To communicate his status 

C. To improve the insulation of the office 

D. To strengthen the office walls 

The response distributions for this item for al! of the American Industry and control group 
students tested in 1966-7 and 1967-6 were: 



Response 


American Industry 


Control 


A 


7.3% 


11.4% 


B 


74.1% 


57.0% 


C 


13.9% 


21.2% 


D 


4.7% 


10.4% 



These data indicate that the control group students were more likely to base their choices 
on the more salient characteristics of materials. (Chi square value is significant at .05 level.) 

The 17% difference is not dramatic but the rcrponses to the item provide some critical insights 
into the students' structure of knowledge. Items could have been written on specific bits of 
knowledge which would have produced larger differences in scores. However, these items would 
not have produced the information needed to develop and revise the curriculum. 

Another index of the strength of an experimental treatment is its abf’ity to produce signif- 
icant effects in subsequent experimental settings. Test results for three ye.;fs, 1966*69, in 
School H are graphically displayed in Figure 6. Each American Industry class was one semester 
long. Before school started each fall, the treatment, the study of American Industry, was rand- 
omly assigned to one class. The other class(as) served as controls. During thy second semester 
the first semester control group (s) studied American Industry and the first semester American 
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Performance on Achievement Test in School 



Industry group studied industrial arts. Both groups had the American Industry instructor during 
each term, however, he taught industrial arts to the control group. During the 1968-69 school 
year, the schedule did not permit a control group 

Statistical comparisons of the achievement test performance of experimental and control 
groups at the end of semester I and semester II in 1966-7 and 1967-8 using analysis of variance 
and analysis of covariance respectively revealed that the American Industry classes gained signif- 
icantly more knowledge during each semester. A t cest of differences on the 1968-69 scores 
disclosed significant gains for both classes. Also, the slopes of the acquisition lines for the two 
classes in 1968-69 are very similar to those for the first two years--e$pecially to the slop:; in 
1966-67. 

Another hypothesis related to the effectiveness or strength of the American Industry 
treatment was than gains in performance on the retention test {semester II administration for the 
students who had American Industry during semester I) should be zero or negative. This hypo- 
thesis was suggested by the findings in learning research that disuse leads to forgetting. In two 
instances, this hypothesis was confirmed with a decrease in performance. In the third instance 
there was a slight increase in performance. 

As noted previously, it was not possible to develop Level II to the same degree as Level I 
nor was there an opportunity to evaluate it in as many settings as Level i. In the seven com- 
parisons made, two (29 %) revealed significant differences in favor of the American Industry 
class, one (14 %) approached significance, and four (67 %) were not significantly different. In 
each comparison, the American Industry groups mean performance was higher than that of the 
corresponding control group. 

Only one of the teacher and school characteristics appeared to have a relationship with 
student performance. Two teachers pestered the Project to be included as participating teachers. 
These men had followed the development of American Industry during its initial phases and were 
determined to include it in their program. Each of these teachers produced statistically significant 
differences in performance on each occasion they taught American Industry. This should not be 
inferred to mean that the other teachers were not interested. However, they did not enter the 
Project in the manner described above. Moreover, several of the teachers invited to join the 
Project also produced significant differencer. 

Follow-up Study . In regard to the rulevance of the information and concepts acquired in 
the study of American Industry, a majority of former students in response to a follcw-up 
survey said that what they learned in American Industry was of high value and another 59.7% 
rated it as of moderate value on their jobs. The remaining of 8.3% said that it was of no 
value. Almost 92% of the responders considered that what they had learned in American Industry 
was of value in their everday activities. (See Appendix M for the report of this study.) 

Relationship Between Ability and Performance 

The high ability groups consistently performed better on the achievement test. In 85% 
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of the comparisons in which ability groups could be stratified, the high ability group performed 
significantly better than the middle and low ability groups. 

Only two interactions between ability and treatment were observed. Both of these were 
disordinal interactions that involved a reversal between high and middle ability groups in the 
American Industry and control groups. In both instances, the middle ability group in American 
Industry out performed the high ability group which had experienced the same treatment. In 
the control group the 'nigh ability group performed better than the middle group. Since there 
were so few interactions, they probably do not reflect a significant relationship between ability 
and course content. 



Summary 

In summary, the results on the Projects' achievement test were somewhat ambivalent. Less 
than 50% of the comparison involved significant differences in performance on the achievement 
test. However, when the comparisons in which the instructor did not follow the Projects' 
instructors gu.de are discounted, the percentage of significant and near significant differences is 
slightly more than 50%. In addition, the effects reported for School H demonstrated that the 
study of American Industry can consistently produce significant effects. Also, the students 
included in a follow-up study of graduates, indicated that what they had learned in their Level I 
American Industry courses was relevant to their jobs and to their everyday activitie.. 

Students’ Perception o f Industry 

In a study conducted during the 1967-68 school year, American Industry students were 
found to have more positive attitudes toward industry than their peers in the control groups. 
These attitudes were reflected in more positive responses to statements such as “I have greater 
appreciation for industry after studying it in class" and negative responses to statements such as 
"products produced by industry are usually rf low quality." (See Appendix L for the detailed 
report of this study.) 

Analysis of the response patterns for each of the items in the attitude toward industry 
scale revealed that the American Industry students exhibited a significantly more positive attitude 
toward industry. Thus, on the basis of these data, the study of American Industry appeared to 
produce a more positive perception of industry. 

Impact on Career Choices and Decisions 

In a study of interests and attitudes influenced by the study of American Industry, Nelson 
(1968) found two Likert items on which the American Industry students had a unique and more 
positive response pattern than the students in the control groups - (1) "I will probably work in 
industry" and (2) "There are many job opportunities in industry that I did not know about 
before taking this class." Approximately 56% of the American Industry students agreed with 
the latter statement and only 18% disagreed. In contrast, 44% of the control group students 
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agreed with this statement and 21.7% disagreed. The remaining students in both groups were 
undecided. Nearly 36% of the American Industry students responded that they would probably 
work in industiy, 33.2% disagreed, and 45.1% were undecided. 

These additional statements directly related to occupational decision making were included in 
the instrument used in this study. On two of three statements * (1) “This class has changed 
my thinking about the work I will do after my schooling has been completed," and (2) "This 
class has started me thinking about the work I will do after my schooling is done," American 
Industry and control groups did not indicate that the two patterns were significantly different. 

Responses to the last statements on occupational behaviors • "Some day I might start a 
business of my own" were very similar for each group. About 33% of the American Industry 
and control group students agreed with this statement, 43% were undecided and 20% disagreed. 
(See Appendix L for the complete study.) 

In the fall of 1969, a follow-up study was conducted. Since the majority of the* students 
associated with the Projec: had studied American Industry while they were in junior hi«,h school, 
the number of high school graduates from the American Industry and control groups was small. 

In total, 121 students from three schools were contacted in this follow-up study. All of the 
American Industry students in this study had enrolled in only the Level I course, during their 
years in senior high school. 

Table 8 presents a summary of the influences reported for educational plans. The :hi 
square value for this set of responses was not significant. It is interesting, however, to note that 
a majority said that their high school courses, American Industry included, had no effect on their 
educational plans. Possibly the student's plans are fairly well determined by the time he enters 
high school or high school experiences have only a minimal effect on his plans. Also, it should 
be mentioned that the students in this sample were probably not individuals who had extensive 
and well defined plans for education beyond high school. The group had average I.Q.'s ;<nd 
were not pursuing the traditional college prep program. In addition, less than 20% entered college 
after graduating from high school. This contrasts with the more than 40% rate of college attend- 
ance reported by the principals in the three schools. Also, as noted in Table 2 the tenth 
through twelfth grade sample from which these students came had more students from lower 
level occupations and slightly fewer from the managerial, professional and entrepreneurial levels. 

Even though relatively few effects on educational plans were reported by American Industry 
students, substantial impact was reported by several students. For instance, one former student 
said that his experiences in the marketing phase of the student enterprise led him to pursue a 
college degree in marketing research. Another responder remarked that he decided to seek, more 
educaticn after he discovered its importance in industry. 

Occupational plans tended to be influenced more by the study of American Industry. (Note 
Table 8). Of those responding to the follow-up study, 56% reported that American Industry 
had an influence on their occupational plans. In seme instances this influence was dramatic. 

For example, one young woman responded that her experiences wording on an assembly line in 
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TABLE 8 



Influence of the Study of American Industry and 
High School Courses on Educational and 
Occupational Plans 



Educational Plans 




Group 




Degree of Influence 


American Industry 


Control 


None 

Some, general 

Specific, Selection of courses or major 


71.1% (47) 
13.6% ( 9) 
15.1 % (10) 


52 (13) 
24 ( 6) 
24 ( 6) 


Occupational Plans 


None 

Gave information on job opportunities 
Basic factor in selecting my vocation 


43.9% (29) 
39.4% (26) 
16.6% (11) 


56% (14) 
32% ( 8) 
12% ( 3) 




an American Industry class were a major factor in her selection of a production line job in a 
manufacturing plant. (A complete report of the follow-up study is given in Appendix M.) 

The data gathered in the follow-up study agreed with the infornniion collected in the 
interests study. Both investigations revealed that the study of American Industry had an influence 
on occupational decisions. 

Influence on Ability to Think Conceptually 



Two separate sets of data were collected in relation to this questions. An instrument with 
pictorial stimuli was designed especially for this purpose. Also, a multiple choice item was dev- 
eloped for the achievement test. The data gathered by these two techniques will be presented 
and discussed in the following sections. 

Pictorial Stimuli 

After considerable analysis of the nature of conceptual thinking and responding, pictorial 
stimuli representing phenomena in industry were selected as the basis for an instrument to mea- 
sure the ability to think conceptually. (See Appendix K for the rationale and description of 
this instrument and the complete discussion of the results.) Four sketches were se'ected for the 
test, illustrating people engaged in activities that could be ascribed to industry. Each sketch was 
general anough to permit the students to project their perceptions into the situation identified. 

In addition, common attributes were included in the four sketches. For example, each con- 
tained at least one person. 

The responders were asked to describe as completely as possible what they saw in each 
sketch. These responses should reflect the influence the study of American Industry has on the 
students' cognitive structure. After they had done this, the responders were asked to define the 
common elements in the four stimuli, it was hypothesized that students who were more orient- 
ed to conceptual thinking would be able to list more commonalities and identify more subtle 
attributes. Responses to these items were scored on the basis of (1) fluency • number of unique 
correct responses, (2) saliency • degree of abstractness represented by the response, and (3) con- 
cept labels - the number of times American Industry concept labels were given in the responses. 

Comparison of the fluency and saliency scores for items one through four disclosed that in 
three of ten comparisons the American Industry classes scored significantly higher. Review of the 
response patterns for each item revealed that most of the patterns favored the American Industry 
groups (A detailed discussion of these results is presented in Appendix K.) Thus, there would 
appear to be some cognitive effect associated with the study of American Industry. Concept 
labels were used somewhat more frequently by the American Industry students. However, this 
effect did not appear to be of much greater magnitude than the saliency and fluency effects. 

No statistic! ly significant differences were found between the experimental and control 
groups on the saliency, fluency and concept label scores for the responses to item five, which 
asked the students to identify the commonalities among the four stimuli. (See Table 9) 
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TABLE 10 



Relevant and Irrelevant Dimensions and 
Values in Concept Item 



Dimensions 



Values 

Critical Value Distractors 



Width 


Narrow 


Wide 


Decoration 


Stripe 


Plain , non-stripe 


Shape of end of tie 


- 


Square , pointed 


Direction of stripe 


- 


Horizontal , diagonal 



In 75% of the comparisons, the American Industry class performed better than the control 
group on this item. None of the differences in performance, however, attained a level of stat- 
istical significance when analyzed with chi square. Overall, 70.2% of the Ameri .an Industry 
students and 62% of the students in the control group answered this item correctly. Although 
there were no statistical differences in performance, the trends would seem to indicate a slight 
effect. 

Results on the pictorial essay test and the multiple choice item did not indicate that the 
study of American Industry had a significant effect on students' ability to think conceptually. 
Trends in the data suggested some effect but either the effect was too small or the data collect- 
ion instruments were too insensitive to measure it definitively. 

Disadvantaged Students Study American Industry 

As mentioned in the ingredients section in this chapter, the Level I American Industry 
course was designed and developed for average and above average students. During the 1965-66 
school year, the American Industry Project had an opportunity to work with disadvantaged groups 
in two schools. The two settings provided a unique contrast. In one school American Industry 
replaced industrial arts in a traditional schedule of courses. In the second, American Industry 
was the core experience for a store-front school. 

The study of American industry met with mixed success in the traditional school. Students 
were enthusiastic about the enterprise and any profits it generated. The student booklets proved 
to be difficult tc read and many of the learning activity guide sheets did not communicate with 
the students. The participating teacher in charge of this group was encouraged to make any 
modifications in the course and instructional materials that he thought would make them more 
effective. However, his schedule and responsibilities did not provide him -with enough time to 
devise many modifications. 
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For this school, the achievement test was recorded on magnetic tape and played to the 
experimental and control groups. In each of the two years the Project worked with this school, 
the American Industry classes scored higher on this test than the control groups. However, these 
differences were not statistically significant. 

American Industry provided the integrating theme for the learning experiences developed in 
the store front school. A small industry was established and the students contracted with local 
businesses to do packaging and assembly work. Approximately half of the school day was given 
to work in the student industry. The remainder of the day was devoted to math, communication, 
and other traditional school subjects. 

The industry was managed and manned by students. Each was paid a wage for his time. 

The tasks and problems encountered while operating the industry were used to stimulate interest 
in American Industry concepts and in traditional subject matter such as math. For example, 
when the student has to calculate his wages based upon piecr ^art rates the value of mathematics 
became very real and importance of the knowledge attributed in the Projects' definition of indus- 
try became apparent. 

Observations by the American Industry Project staff and feedback from staff members at 
the store-front school indicated that the American Industry s+.ucture and the enterprise activity 
were an integral part of this successful educational innovation. The store-front school project 
drew national attention and recognition. 

On the basis of the experience with these two groups of disadvantaged students, the Amer- 
ican Industry Project staff concluded that the concepts and structure of knowledges were relevant 
to educational experiences for the disadvantaged. Also, it was determined that a special instruct- 
ors' guide, new reading materials, and additional learning activities were needed for successful 
programs for this group. 

Additional Impacts of the American Industry Project 

Although the changes in students' knowledges and perceptions reported in the previous 
section are ot prime concern and interest, the American Industry Project has also had some mark- 
ed effects on the staff members involved in the Project and on industrial arts curriculum?. 
Examples of these effects will ba presented in the ensuing paragraphs. 

Participating Teachers and Project £+aff Members 

Participation in the development of the American Industry courses brought attention and 
recognition to the participating teachers. For example, 80% of the participating teachers reported 
visitors to their American lndus'.;y classes during the 1967-68 school year. During this same year, 
45% reported that they had been asked to make presentations on American Industry. Since this 
time, several of the American Industry teachers have served as resource people at workshops and 
institutes, instructors for extension courses in American Industry, and speakers at conventions. 
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Two of the participating teachers decided to study for doctor's degrees during their tenure 
with the Project. In addition, two more decided to start work on their masters' degrees. Two 
of the participating teachers joined the Project staff at Stout State University. 

During discussions with the participating teachers .almost all of them indicated that their 
experiences with the Project had been very worthwhile. Some commented that they had gained 
insight into curriculum development that they would not have achieved without this experience. 
Others noted that their association with the Project had improved their teaching. 

Three of the eight staff members of the American Industry Project decided to study for 
doctor's degrees during the time they were with the Project. > A .M three have successfully pursued 
these studies. One has completed his doctorate, another will finish during the summer of 1971, 
and the third has completed most of the course work. All of the staff members authored 
journal articles related to their activities on the Project, and most made presentations at national 
conventions such as American Vocational Association, American Industrial Arts Association, and 
American Educational Research Association. 

Without doubt the American Industry Project had a significant impact on the teachers and 
staff members associated with it. It stimulated marked professional growth and enlarged the 
horizons of these men. 



Impacts on Stout and the National Scene 



Stout State University 



A variety of changes and innovations at Stout have bean stimulated in total or in part by 
the American Industry Project A new degree program to develop American Industry teachers 
was approved in 1966. Shortly after the bachelor's degree was approved, a special emphasis in 
American Industry was included as an option in the master's degree in Industrial Education. 

Also, a new graduate course entitled "Structuring Knowledge" was deigned on the basis of the 
Projects' experiences in developing the conceptual structure on the knowledges of industry. 

At the present time, the School of Applied Science and Technology is studying plans for 
restructuring its departments and the disciplines taught in these departments. The goal of the 
restructuring is to provide a broad and conceptual study of industry. Thus, it is apparent that 
the American Industry Project has had a marked impact on Stout State University. 

National Scene 



During the four years the American Industry Project was funded by the USOE, more than 
3000 individuals asked to be placed on the Project's mailing list. In 1968, 198 reader's of the 
Project's newsletter wrote for copies of two papers announced in the spring edition. These people 
resided in 37 states, 1 territory, and 2 foreign countries. (See Appendix for the complete study.) 
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An analysis of the articles published in IAVE i n 1958, 1963, and 1966 revealed an increased 
use of the American Industry concept labels. Numerous visitors stopped in Menomonie to visit 
with the Project staff. And, papers on various aspects of American Industry were presented at 
a variety of national conventions. 

On the basis of the curriculum changes at Stout and requests for information and present- 
ations, it would appear that the American Industry Project had a significant influence on the 
local and national scene. 



Summary 



Evaluation activities in the American Industry Project were 
ingredients, ( 2 ) processes, and (3) products. Analysis of the ingr 
American Industry instructional materials were valid and logically 



Projects' conceptual structure of industry. Information -• 
cated that students in the American Industry classes re* 
attending secondary schools. 



rried on in three areas: (1) 

• mts data revealed that the 
ii'** .red in relation to the 



mI ty 3 ’ ocio-economic status indi- 
r « tion of students 



Feedback from teachers and Project staff member, 
courses were not always taught as designed. In most : 
ials were used in a manner appropriate to their original u 
course plan and the intent of the instructional materials Icr. 



American Industry 
r , the instructional mater- 
CJ * ficant deviations from the 
red student performance. 



In approximately half of the settings in which the instructional materials were utilized 
according to their design, the American Industry group scored significantly better than or approach- 
ed scoring significantly better than (IO^P^.05) the control group. In only one of more than 
fifty comparisons did the control group mean on the achievement test exceed the mean for the 
American Industry group involved. Study of American Industry students' responses on an opin- 
ionnaire and in a follow-up study disclosed that the study of American Industry had some sign- 
ificant effects on career choice and the students' knowledge of job opportunities in industry. 



Additional impacts of the American Industry Project were observed throughout the duration 
of the Project. Project staff members were encouraged to pursue advanced study. Increasing 
numbers of references to American Industry appeared in the professional i terature and the con- 
cepts in the conceptual structure of industry became more prominent in journal articles. 
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CHAPTER III 



SUMMARY AND RECOMMENDATIONS 



Summary 



In 1962, a group of Stout State University professors formed a study group to investigate 
the status of industrial arts and to consider its role in the education of students. Their efforts * 

and ideas provided the basis for the American Industry Project. A Four year research grant from 
the USOE supplied the resources to complete the conceptual structure of knowledges started by 
the study group. Also, during this grant, three high school courses were developed, evaluated, and 
refined. 

The conceptual structure was revised and refined through visits to industry and schools of 
industry. Later it was validated through the comments and reflections of business and labor 
leaders. 

Three courses were constructed for secondary school students: (1) Level I for eiohth grade, 

(2) Level II for ninth and tenth grades, and (3) Level III for eleventh and twelfth graders. 

The evaluation system designed for the Project was integrated with the course development 
process. Information provided by the evaluation was utilized as a basis for making a variety of 
curriculum development decisions. In addition, data from the evaluation were used to determine 
the effects of the study of American Industry. 

Over 3800 students participated in the evaluation during the three years it was carried on. 
Approximately 59% of this number studied American Industry. Eighth grade boys comprised * 
about one-half of this sample. Ninth and tenth grade boys were the next largest segment. 

The occupations of the fathers of the students encompassed in the evaluation were very 
similar to those found in a national sample of students. Ability scores reported for the partici- 
pating students ranged over the complete spectrum found in secondary schools. 

With one exception, all of the participating teachers had prior experience teaching industrial 
vts. Most (71%) had master's degrees. All had an interest in manipulative and physical activ- 
ities. 



A theoretical-empirical approach was employed in developing the instructors’ guides. Gagnefe 
work on learning heirarchies and the Project's conceptual structure of knowledges provided the 
theoretical bases for designing the guides. Field testing and feedback from the teachers produced 
the empirical basis for revising the guides. The first level guide was revised three times and the 
second level guide was reworked once. All of the remaining instructional materials were corre- 
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iated w : '!i the guides and evaluated during the fiala tests. 

Although it was not possible to select participating schools and teachers were verv similar 
to those from a sample representative of all secondary schools in the United States. On three 
variables that have been found to have significant impacts on student performance (1) salaries for 
beginning teachers, (2) years teaching experiences attained by the school staff, and (3) average 
per pupil expenditures, there were no marked differences between the American Industry schools 
and the national sample. Also, it was found that the distribution of occupations of students' 
fathers was similar to that of a national sample of students. 

As noted previously, all of the instructional materials were evaluated as a part of the evalua- 
tion system designed for the Project. Results of the teachers' feedback indicated that a detailed 
teachers guide with correlated instructional media and student materials were essential to an 
effective course. The student enterprise activity was highly rated by the teachers and students. 
Also, a need for more activities and methods for utilizing laboratory activities in a manner that 
contributed to the acquisition of concepts was apparent. 

An achievement test was written to supply feedback on cognitive outcomes and to provide 
a basis for comparing experimental (A.I.) and control groups. In those settings in which the 
teacher followed the instructor's guide or its intent closely, approximately 40% of the American 
Industry classes scored significantly higher on the achievement test than the control group. In 
another 12.5% of the comparisons, the F value for the analysis of variance was between .10 
and .05. Again these near misses favored the American Industry classes. None of the control 
groups scored significantly higher than their American Industry counterparts. And, in only one 
experimental setting did the control group mean on the test exceed that of the American Ind- 
ustry class. Analysis of the results of the achievement testing revealed that semester classes did 
not produce as many significant differences as courses which spanned the entire school year. 

In a study of interests and attitudes conducted during the 1967 • 68 school year, it was 
found that American Industry students exhibited a more positive attitude toward industry. In 
addition, the responses of the American Industry students gave evidence that their study of 
American Industry had an impact )n their knowledge c 4 occupational opportunities in industry 
and their interest in working in industry. A follow-up study conducted in 1969 with the first 
group of high school graduates who had studied American Industry (all members of this sample 
had enrolled in only the Level I course) obtained responses which substantiated the effects the 
study of American Industry had on occupational behaviors. 



Recommendations 



Based on the experiences and knowledges accured during the development and evaluation of 
the American Industry secondary school courses the following recommendations are made. 
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Continue refinement of the structure of industria' knowledges. 

Review Level I and consider the following revisions. 

a. Add a unit on concept learning to make explicit to the student the 
mode of thinking and learning required. 

b. Include career exploration activities as a formal aspect of the course. 

c. Revise Unit II - History of Industry. 

d. Incorporate additional material on the environment of industry, espec- 
ially the societal context of industry. 

e. Revise the student booklets to include more photographs and illustra- 
tions and change the names of the characters in the story. 

f. Develop additional activities and alternatives to the enterprise. 

Refine Levels II and III using the procedures developed for Level I. 

Refine the procedures for identifying critical phenomena in industry and 
analysing them into logical elements of knowledge and performance such as 
concepts and manipulative operations. 

Further define procedures and processes for moving from a theoretical structure 
of knowledges to the design of instructional programs and materials. 

Develop protocols for American Industry teachers to utilize as they instruct in 
the affective and cognitive domains. 

Investigate in further detail what is learned from psychomotor activity. 
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APPENDIX A 



Introduction to Appendixes 



Several appendixes were included in this report. This was done to provide in-depth discus- 
sions of specific problems, tasks, and research without making the text of this report overly long 
and tedious to read. Instead, it was decided to provide essential details in the text and place 
the remainder in the appendixes. 

The author of each appendix is noted on the title sheets. Since several writers were resp- 
onsible for these papers, style and form varies to some degree from one appendix to another. 

In order to preserve the thoughts and work of each writer only slight editing was done. 

Each appendix is paged separately. A letter code is used to identify each appendix and 
arabic numbers are used to identify pages to the appendixes. Frequent reference is made in 
the text of the report to the appendixes. These references correlate the text with the append- 
ixes that follow. 
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In 1962 a group of faculty members at Stout State University gathered to discuss the con- 
tent and objectives of industrial arts. This group evolved into an organized seminar which init- 
iated the first steps that were to lead to the conceptual structure of industry which provides the 
basis for the study of American industry. Out of the effort of this seminar grew the project 
funded by the Ford Foundation and the American Industry Project funded by the U.S. Office 
of Education. 

The original conceptual structure of the knowledges of industry is given in Figure 1. Two 
of the primary inputs into the development of this structure were the theoretical literature re- 
lated to the attributes and procedures for constructing conceptual structures and the literature on 
concept learning. Using this information as a framework and guide, the members of the Project 
and the participating teachers searched the literature on industry and made a number of contacts 
wi th peopie in industry to acquire the information required to delineate this initial structure. 

Three products resultea from these activities: (1) the major concepts of industry, (2) the struc- 

ture of knowledges, and (3) a model for each major concept which identified the sub-concepts in 
the area. 

This first structure was then used as a base for the development instructional materials 
which were tried out in selected secondary schools during the ensuing academic year. At the 
end of this year the participating teachers and the Project staff, which had now grown to seven 
members, reviewed the structure and the materials. After corrections were made and the defini- 
tions of concepts were revised and refined, a packet was developed for presenting the concepts, 
concept models and the structure to people working in industry. Interviews were scheduled with 
a number of individuals connected with industry and with selected university staff members in 
several universities around the country. Project staff members presented the materials to the per- 
son being interviewed and discussed them in some detail with him. Typically only one concept 
was reviewed with any one interviewee. However, in some cases where the person occupied a 
relatively high position in the company, for example, the Vice President for Research, two or 
more concepts may have been discussed. The interviewer did more than place the materials in 
front of the person being interviewed. Typically, i.*j would present an overview of the project 
and discuss the conceptual structure. In addition, it was found that the most productive sessions 
developed when the interviewer had a set of qi jstions based upon the structure that could be 
used to stimulate the discussion. 

As a result of these interviews the concept of public interest was dropped from the original 
conceptual structure of industry. This action was based upon the comments from a number of 
the people in industry, management and labor included, that conveyed the idea that public in- 
terest was more related to the environment within which industry operated. At this point two 
environmental rings were added to the conceptual structure. (See Figure 2 ) The main input for 
this action resulted from the perceptive comments of Richard Thornbury, Division Director of the 
Industrial Relations Center at the University of Chicago. 

In the spring of 1966, the input from the various consultants was summarized by the Cur- 
riculum Specialist and presented to the Project staff. Then each concept was studied in-depth 
with task force teams comprised of members of the project given responsibility for revising selected 
concepts. Reports from each of the task forces were presented to the total Project staff and 
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final revision were determined after the reports had b»;en discussed. Although the project con- 
tinued to acquire information on its concepts and the conceptual structure, the structure delin- 
eated in the stage just described served as the basis for most of the instructional materials dev- 
eloped during the last three years of the Project's grant. ’ (See Figure 3) 

Once the final revision of the structure had been completed, the Research Specialist sug- 
gested that the structure, the conceptual models, and the definitions of the concepts be presented 
to a representative group of leaders in industry and labor in order to validate them. New pack- 
ets of materials, overhead transparencies, and a standard presentation were prepared to present a 
conceptual structure to this audience. Groups of twenty to thirty individuals were identified and 
assembled in each of the following regions: Eastern, Upper Midwest, West, and South. A 

standard presentation was made to each group. (An outline of the presentation is given in Ap- 
pendix B-A) The same team did not make each presentation. After the presentation and dis- 
cussion had been completed each participant was asked to complete the American Industry 
Inventory Form. The items in this form are presented in Table 1. 

The American Industry Inventory Form was designed around the significant questions assoc- 
iated with the development of the Conceptual Structure of the Knowledges of Industry. State- 
ments related to the definition of industry, the degree to which the structure reflects industry, 
and the utility of the structure were presented in Likert format. The responder had six possible 
responses ranging from strongly disagree to strongly agree plus an N response to be used if a 
particular area had not been discussed in the presentation. 

Preliminary analysis of the data indicated that there were no significant differences between 
regions, thus, the data were pooled and the summary presented in Table 1 is based upon all of 
the responses. One of the questions that often has been raised to the Project was concerned 
with the breadth of the Project's definition of industry. Responses to Item 1 indicate that the 
Project's definition is valid. On Item 1, 80% of the respondents agreed that the definition is 
appropriate. Only 11.5% disagreed with the definition. A considerable amount of discussion and 
input from various sources led to the inclusion of the term profit in the American Industry def- 
inition of industry. Responses to Item 3 indicate that 88% of the respondents agreed with that 
action and about 9 , i disagreed. On the basis of the responses to the first three items, the 
American Industry definition of industry appears to be valid,. 

The project's review of psychological literature indicated that a conceptual structure lent 
efficiency to learning. However, if the structure is to be valid it must accurately reflect the 
phenomena chat it is reported to represent. The response pattern on Item 4 indicated that 73.4% 
thought that the conceptual structure gave an accurate summary of the major ideas used in in- 
dustry. About 10.6% disagreed with this statement. Another item related to this is Number 0 
which states that the conceptual structure of the knowledges of industry gives one of the best 
representations of the ideas basic tc a successful business that I have seen. The responses to this 
item indicated that 63.7% agreed with this while 8% disagreed. Items 5 and 7 are also related to 
the degree to which the structure reflects the knowledges and activities of industry. The structure 
depicts a relationship between a given concept and every other concept in the structure. Re- 
sponses to Item 7 indicate that 61% thought that the American Industry structure was valid; 11.5% 
thought that not all of the concepts were related to each other. The response pattern on Item 5 



B-4 




O 

ERLC 




Figure 3 

A CONCEPTUAL STRUCTURE 
OF THE KNOWLEDGES NECESSARY 
TO UNDERSTAND AMERICAN INDUSTRY 



B-5 

94 



O 

ERIC 



TABLE I 



Summary of Business and Labor 
lo I he 

American Industry Inventory 



1. Your definition of industry is 

appropriate 

2. Your definition of industry is 

too narrow 

3. There is an overemphasis on 

profit in your definition of 

industry 

4. The conceptual structure of 

the knowledges of industry 
gives an accurate summary of 
the major ideas used in in- 
dustry 

5. A knowledge of the interre- 

lationships shown in your 
structure is important in 
modern business 

6. Some of the concepts in your 
structure have no relationship 
to the management concept in 
actual business practice .... 

7 . In actual practice , some of 
your concepts have no interre- 
lationship with the other 
concepts on the structure . . 

8. The conceptual structure of 

the knowledges of industry 
gives one of the best repre- 
sentatations of the ideas 
basic to a successful business 
that I have seen 

9 . Your models provide a useful 

tool for understanding each 
concept area 

10. It is a good idea to discuss 

the content of each model with 
people in industry who are 
knowledgeable in the specific 
areas 



SD 


D 


V 


A 


SA 


N 


NR 


.9* 


10.6 


4.4 


66.4 


13.3 


0 . 


4.4 


6.2 


47.8 


18.8 


18.6 


2.7 


1.8 


4.1 


27.4 


50.4 


6.2 


8.8 


.9 


0 . 


6.3 


.9 


9.7 


10.6 


62.8 


10.6 


.9 


4.4 


0 . 


1.8 


2.7 


58.4 


31.9 


.9 


4.4 


8. 


51.3 


18.6 


8.8 


1.8 


2.7 


8.8 


6.2 


54.9 


18.6 


10.6 


.9 


.9 


7.8 



1.8 6.2 2 L 2 54 . 9.7 1.8 5.3 

.9 4.4 9.7 67.3 8.8 3.5 5.3 

0 . .9 0 . 42.5 50.4 1.8 4.4 
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TABLE l (Continued) 



11. 


Present day literature will be 


SD 


D 


V 


A 


SA 


N 


NR 




of as much help as people in 
indus*rv in providing ideas 
for the models and your course 
content 


1.8 


39.8 


18.6 


25.7 


5.3 


.9 


8. 


12. 


Industry people will be the 
best source of ideas for your 
models and content 


0. 


9.7 


11.5 


47.8 


23.9 


0. 


8. 


13. 


Knowledge of the ideas in- 
cluded in your structure would 
be of value to all employees 
in industry 


.9 


8.8 


7.1 


54.9 


22.1 


.9 


5.3 


14. 


An understanding of your con- 
cepts of industry should make 
an employee more productive . 


2.7 


8. 


18.6 


46.9 


15. 


1.8 


7.1 


15. 


Most entering employees al- 
ready have an adequate under- 
standing of the knowledges in 
your structure 


38.1 


53.1 


1.8 


1.8 


1.8 


0. 


3.5 
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supports the value of the relationships shown in the Project's conceptual structure of the knowl- 
edges of industry. Over 90% agreed that a knowledge of the interrelationships is important in 
modern business. In summary the responses to Item 4, 5, 7, and 8 indicate that the conceptual 
structure is a valid representation of the phenomena of industry. 

In order to gain a greater insight into the area defined by each of the major concepts in 
the structure a conceptual model was delineated for each of the conceptual areas. Item 9 in the 
Inventory asked the responders to evaluate the utility of these modi Is. Approximately 76% found 
that they were a useful tool. About 5.3% did not think that the models were useful. 

In addition to having a valid representation of the knowledges of industry the Project was 
concerned that it develop a structure that would have utility to the person who comprehended it. 
Items 13 and 14 in the Inventory relate to this consideration. Seventy-seven % of the responders 
thought that the knowledges encompassed in the structure would be of value to all employees in 
the industry. About 10% of the respondents disagreed with this statement. On Item 14 about 
72% of the responders thought that an understanding of the American Industry concepts of ind- 
ustry would make an employee more productive. Approximately 11% disagreed with that state- 
ment. The high level of favorable response to Items 13 and 14 indicate that there is consider- 
able utility in the conceptual structure developed by the American Industry Project. 

Up to this point the validity of the conceptual structure of the knowledges of industry is 
apparent, however, if the development and implementation of instructional programs based upon 
these materials is to be justified there must be some evidence to the effect that there is a need 
for learning the knowledges encompassed in the conceptual structure. Responses on Item 15 
indicate that there is such a need. Over 91% of the responders disagreed with the statement that 
most entering employees already have an adequate understanding of the knowledges in the structure. 

In summary, responses on the American industry Inventory indicated that (1) the Project's 
definition of industry was appropriate, (2) the Conceptual Structure of the Knowledges of In- 
dustry was an accurate reflection of industrial phenomena, (3) the structure and concepts were 
important to understanding industry, and (4) the knowledges encompassed in the structure are not 
held by most entering employees. Thus, it appears that the American Industry Project's definition 
of industry and its conceptual structure are valid and possess a considerable degree of utility in 
making the phenomena of industry comprehendable. In addition, there appears to be a need to 
develop these knowledges in our students. 

The activities described in this paper represent the initial steps in validating ihe Project's 
structure. Final validation rests upon the influence a conceptual knowledge of the structure has 
on the learner's understanding of industry. 
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Appendix B-A-l 



Outline of 



Industry - Labor Presentation 



INDUSTRY-LABOR PRESENTATION 



Slide No. 


Contents (or Title ) 


Comments 


1 

2 


Blank 

American Industry Inventory 


Discuss how to complete the 

form 


3 


Machine Shop 


Stout’s Curricula 


4 


Plant planning 




5 


Packaging 




6 


Fabrics 




7 


Art 




8 


Blank 


Introduce some of 






Stout’s instructional 


9 


Ampex tape deck 


technology 


10 


CCTV 




II 


Blank 


Introduce the American 
Industry Project 


12 


Title Slide “ The American 
Industry Project ” 




13 


Change needed , 1962-63 


Faculty meetings 


14 


Project; 1963-64 


Ford grant 


15 


Project , 1964-65 


USOE grant 


16 


Project, 1965-66 


Development of materials and 
20 secondary schools for 

tryout 


17 


Blank 




18 


Definition of American Industry 


Read definition 


19 


General Objectives of American 

Industry 


Read objectives 
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In du s try -La bar l*resenla tio ns 


Slid*’ No. 


Contents (or Title) Comments 


20 


Blank Introduce the need for 

defining our terms 


21 


Guide and hitching post 


22 


Concept Formation: Show how a concept of 

“It’s Bedtime” lime develops 


23 


“ It's Suppertime ” 


24 


“It’s Time To Get Up” 


25 


“ Sometime ” 


26 


“Tomorrow ” 


27 


“ Hurry Up ” 


28 


“It’s Time To Go To School ” 


29 


“ You’re Late For School” 


30 


“30 Days Hath September” 


31 


“ August i8” 


32 


“ Show Starts in 15 Minutes” 


33 


“ Tempus Fugit” 


34 


“ Youm Get 75i An Hour” 


35 


“3 Days Till Friday” 


36 


“There Goes The Bus” 


37 


“I’m Late To Work Again” 


38 


“ Breakfast in New York, Lunch in London” 
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Industry -Labor Presentation 



No. 


Contents (or Title ) 


Comments 


39 


“ 1/10 of a Second ” 




40 


“One More Orbit to Re-Entry ” 




41 


“TIME” 




42 


Definition of a concept 


Explain attributes 


43 


American Industry Structure 


Note concepts and inter- 
relationships 


44 


Concepts employed by a carpenter 


Note how concepts are 
used in concert 


45 


Concepts employed by a research 

specialist 




46 


Blank 




47 


Definition of American Industry 


Emphasize the attributes 


48 


American Industry Structure 


Comment on how all in- 
dustries are included 


49 


Sources of information for structure 


Note each 


50 


Sources of information for models 


Note each 


51 


Production model 


Models further define 
each area 


52 


Building the curriculum 


Rationale and structure 
provide the basis for the 

curriculum 


53 


Blank 


Comment that there are 
three levels in the second- 
ary curriculum 


54 


Primary objectives - Level I 


Give each objective and 
note how they fit to- 


55 


Primary objectives - Level II 


gether 
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Indus try- Labor Presentation 

Contents ( or Title ) Comments 



56 


Primary objectives - Level III 




57 


Unit 1 - Industry Tod'iy 


The contents of the first 
level course . Emphasize 


58 


Unit 2 - The evolution of industry 


how the instructional 
cycle brings the students 


59 


Units 3, 4, & 5 


in contact with each con- 
cept several times 


60 


Unit 6 , The Future of Industry 




61 


Unit 7, The Student Industry 




62 


Blank 


Introduce the develop- 
ment of the first level 

materials 


63 


Development of first level 

materials 


Sequence of activities 


64 


First level booklets and 
Instructor’s guide 


Products 


65 


Blank 




66 


Concept of forming - forming roll 


Referents for a concept 
of forming 


67 


die casting 




68 


pan break 




69 


bending 




70 


riveting 




71 


Student planning 


Students participate in 
planning to the degree 
their competencies allow 


72 


Student-Teacher discussion 


Sample learning activities 



B-A-4 







10 Z 



Page 5 

Industry-Labor Presentation 



Slide No. Contents (or Title ) 



73 


Lecture 


74 


Field trip 


75 


Blank 



Comments 



76 


Consultant response form 


Data collection forms 
used in evaluating the 


77 


Test question 


Project 


78 


Test question 




79 


Blank - END 





80 Have the participants complete the American Industry Inventory. 

81 Collect the completed forms. 
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DESIGNING AND MANAGING THE 
AMERICAN INDUSTRY EVALUATION SYSTEM 



Introduction 

During the past three and one-half years, the American Industry Project has been funded 
under Section 4C of the 1963 Vocational Education Act to develop a set of curriculum mater- 
ials for secondary schools and institute a teacher education program for this curriculum. My 
comments today will be concerned with the evaluation system developed for the Project, some 
of the techniques utilized to design it, and a sample of the management tactics used to monitor 
and control the system. 

Keep in mind that this system was designed for a specific project. Thus, in its present 
form, the evaluation system may not be appropriate for other projects or applications; however, 
the principles and concepts used in its development should be of value to those concerned with 
the construction and evaluation of curriculum materials and learning systems. 

The American Industry Evaluation System 

One of the primary purpose of the American Industry Project was to develop a curriculum 
based on the study of industry. Industry as used in this context is a generic term. It includes 
the various businesses and enterprises which produce and market goods or services to satisfy the 
needs of their customers and attain a profit for themselves. The American Industry secondary 
school courses are designed to be transitional subjects between general and vocational education. 
Their objective is to involve the student in learning experiences which asist him to make appro- 
priate vocational decisions, to comprehend his role In our industrial society, and to be a product- 
ive member of society. 

In order to understand the vast number of activities and knowledges utilized by industry, 
the conceptual structure of the knowledges of industry depicted in Figure 1 was developed and 
refined. The basic elements in this structure are the concepts of industry. Interrelationships 
between the several concepts complete the structure. On the inner ball are listed the concepts 
used directly by industry to achieve its objectives. Concepts in the ring represent the environ- 
ment in which industry operates. These concepts interact with those on the ball. 

Four basic rules were established for designing the American Industry courses. First, these 
courses focus on the study of industry. Second, the structure of industry shown in Figure 1 
provides the basis for this studv. Third, inherent in these courses is the study of the concepts 
of industry. And, fourth, provisions for problem solving experiences are included in the learning 
activities. 

Three secondary school courses are being developed on the basis of the structure of the 
knowledges of industry and the four rules just listed. In the first course, the student is introdu- 
ced to rpe basic concepts of industry and has th*» chance to solve some simple industrial problems. 



C-1 




O 

ERIC 




Figure / 

A CONCEPTUAL STRUCTURE 
OF THE KNOWLEDGES NECESSARY 
TO UNDERSTAND AMERICAN INDUSTRY 
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The second course presents each concept in greater detail and depth. Sub-concepts have been de- 
fined for each concept and these provide the structure and resources for the second course. In 
addition, the student becomes more involved with the actual sequences of events and activities 
found in industry and he has the opportunity to solve more complex problems. The third 
course is designed for individual study. Each student is encouraged to select a suitable problem 
and to then study it in terms of the conceptual structure of industry. 

The first course is being developed for the seventh and eighth grades. Target population 
for the second course is grades nine and ten. The third course would be open to eleventh and 
twelfth grade students. Both boys and girls can enroll. 

During the past three years, the American Industry Project staff has devoted part of its 
time to delineating and refining the structure of the knowledges of industry. In addition, a 
sub-group has been working on the three secondary courses just described. The research spec- 
ialist has been a member of this second group and the evaluation system described in succeeding 
paragraphs was developed to meet the needs of these people. 

Two general objectives guided the distillation of this evaluation system. First, it had to 
provide research data related to the effectiveness and outcomes of the instructional materials, 
learning activities, and instructional sequences used in the American Industry courses. Second, 
the information required for the management of the curriculum development activities and the 
evaluation also had to be provided. Fortunately, these are compatible objectives. ’In fact, as 
will be seen later, they complement each other quite well. The evaluation domains and the 
researcn and the development strategy presented in the ensuing comments were designed to 
attain these objectives. 



Evaluation Domains 



Data are collected in three domains in the evaluation system. These domains are entitled 
ingredients, processes, and products. The ingredients domain includes all of the input variables 
in a learnin j situation. Some typical inputs are the nature of the teachers, the characteristics 
of the students, and the quality of the instructional materials. 

As a unit of instruction or course is being taught, a number of teaching-learning processes 
interact with the ingredients and with each other. Teachers try out new materiel and ideas 
with students who have diverse abilities and interests. Students encounter new concepts and 
activities. And, the teacher's colleagues may react to the new curriculum. These activities and 
interactions are encompassed in the processes domain. This domain is concerned with the on- 
going activities and responses associated with teaching and learning within the school environment. 

The last domain is comprised ot the outcomes or products of the studv of American Indus- 
try, This domain is concerned with the characteristics of the student at the end of the course, 
the impressions of the teachers, the reactions of administrators and staff members at the partic- 
ipating schools, and opinions of the parents. Follow-up data are also a part of the products 
domain. 
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These domains are presented in graphic form in Figure 2. The horizontal lines connecting 
the three domains symbolize the interrelationship that exists between them. Moreover, they por- 
tray an interaction and sequence of events that is essential to establishing causal relationships. 

These relationships are especially vital in continuing developmental work. Without the reasons 
for what happened, it is very difficult to make modifications and refinements in curriculum 
materials and procedure*. 

A look at some of the data collected in each area may help to clarify the scope of each 
domain. In the ingredients domain, for example, the structure and sequence of instructional 
units is ascertained and the reading level of the student materials is measured. Participating 
teachers fill out biographical data forms and the students complete information forms. Students 
also complete a questionnaire based on Project TALENT'S Student Information Blank. Guidance 
counselors furnish ability scores for students, and administrators provide statistics on their school 
system. Data on the home communities of the participating schools are secured from documents 
and observation. 

Processes are evaluated through the use of opinionnaire forms, observation, and communication 
with the participants. Teachers react to each lesson a standard rating sheet placed at the end of 
each lesson and unit. Students express their opinions on a rating form devised for this purpose. 

In addition, the Project's supervisor of participating teachers makes regular visits to each school 
to observe the use of the materials and the progress of the course. He observes classes and dis- 
cusses the courses with the students, participating teachers, and administrators. Other members 
of the Project staff also visit the schools as the opportunity arises. 

Another source of processes data is the condition of the instructional materials supplied 
by the Project. For example, booklets thut are clean and have tight bindings indicate that they 
were used little, if at all. 

Outcomes from the study of American Industry are measured in several ways. Students 
complete an achievement test which measures their ability to apply the concepts of industry to 
situations and problems found in industry. They also complete an opinionnaire which provides a 
measure of their attitude toward industry. Reports, written and verbal, by the participating 
teachers give* evidence of changes in their thinking. Even mo revealing are the curriculum 
and laboratory plans they develop after participating in the program. Effects on the schools are 
noted by changes in curriculum expansion of American Industry offerings, and requests for infor- 
mation to the Project. 

The relationship between the three domains can be seen in the following example involving 
the evaluation of the student booklets produced by the Project. 







ERiC 



108 



INCREMENTS PROCESSES PRODUCTS 








c.s 



1C9 







Evaluation Domains in the American Industry Project 



Ingredients 



Processes 



Products 



Reading level of each booklet 
(Dale-Chall formula) 



Teacher description of the Student achievement 
use of the booklet (Feed- (Project's test) 

back form) 



Reading ability of the students 
(Reading tests) 



Supervisor of participating Student opinions of 

teachers observes classroom reading level and contents 
use of booklets and questions (Rating form) 
students 



Interests of the students 
(Student Questionnaire-adapted 
from Project TALENT'S SIB) 



Students indicate how much 
they read (Rating form) 



Concepts in the booklet (Based 
on the Project's structure of 
industry) 



Research specialist reviews Teacher opinions of 

signs of usage on individual booklets (Feedback forms) 
booklets. 



If a student says that a booklet is difficult to read, his reading score can be compared 
with the reading level of the booklet. Then his interests can be considered. Also, the use 
made of the booklet by the teacher can be included as another factor. 

During the past three years, approximately six thousand students have been entered in the 
Project's data file and about six million bits of information have been collected. Arrangements 
were made with the administration in each participating school for the data collection procedures, 
the types of information to be collected, and the security measures to be employed by the 
Project. In brief, the Project guarantied that it would not name individual students in its report 
and that it would not connect specific schools or teachers to results in its reports without prior 
approval of the schools and people involved. 

The information collected is channeled to the research specialist for analysis, interpretation, 
and storage. Most of this information is relayed to a key puncher and is placed on tab cards. 
The original document is retained and the data card is filed for further processing. 

Data gathered in the evaluation are used to make a number of comparisons. For example, 
differences in achievement and attitudes of American Industry students and control group students 
are analyzed. Teacher evaluations of new instructional materials are contrasted with their ratings 
of the materials replaced, and with other instructional resources used in the course. Experimental 
designs are used when possible; however, in some schools quasi-experimental designs have to be 
employed. Within school comparisons and interpretations of results are made within the context 
of the design used. Since schools were not randomly selected, nor formal between school 
comparisons are made. 
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The fact that the Project is continuing its development work over a period of years has 
provided the opportunity to analyze data gathered during several school terms. Increments or 
decrements in performance can be noted on the basis of previous results in the same setting. 

In addition, the effects of changes in instructional materials and learning activities can be asses- 
sed by their influence on performance trends 

To a large extent, the information collected and the results of the comparisons are germane 
to the work of the curriculum specialist. Hence, the research specialist has worked closely with 
him to determine his information needs. Then as the data are analyzed, the curriculum spec- 
ialist receives the results and discusses them with the research specialist. 



Rese f ’ ch and Development Strategy 

As noted previously, one of the basic objectives of the American Industry Project was to 
develop and evaluate curriculum materials for American Industry courses. This objective has 
structured the research and development strategy designed for use with the evaluation domains. 
Although the evaluation domains identified in Figure 2 suggest specific sources of data, decisions 
must be made on the segments of the course to be evaluated, the types of information to col- 
lect, and the data collection schedule. 

Figure 3 gives a graphic representation of the research and development strategy employed 
At the left is a representation of the Project's structure of the knowledges of industry. This 
structure together with its accompanying rationale defines the guidelines and the boundaries 
within which the research and development work is carried on. The solution axis is indicated 
by the horizontal line enclosed in the spiral line which depicts actual results. 

The successive spirals, each with a smaller radius from the solution axis, symbolize two 
essential elements in the strategy. Instructional materials, evaluation techniques, and other products 
are developed and approved on the basis of an evaluation of how they function in the setting 
for which they were designed. The successive spirals indicate that this will probably require 
more than one trial, and that the results from the prior trial will be the basis for the revisions 
for the succeeding trial. In addition, the decreasing size of the spirals indicates an improvement 
in performance. This denotes the value of appropriate feedback from the preceding spiral and 
effective use of this information. 



Progress toward the maximum solution during a spiral becomes less for each succeeding 
cyc!e. In other words, improvement becomes more difficult as the research and development 

cycles progress. The relationship between the distance traversed by each successive spiral is 
given by X>Y^Z. 

The implications arising from this relationship are very important. Given a fixed amount 
of human and financial resources, progress will be less marked in later research and development 
cycles. Hence, operating under the usual conditions, in which these two commodities are scarce, 
required judicious selection of the curriculum programs and products to be further developed and 
evaluated. 
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Fig. t 1 Specific Research and Development Spiral 
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In solving this problem on the American Industry Project, systems concepts were applied to 
the curriculum development work. An inventory was made of the components of the system 
and their interrelationships. After this, the problems associated with these components and 
interrelationships were reviewed to determine how critical they were in terms of reaching the 
Project's goals and the ramifications of solving or not solving them. This gave a basis for 
attacking a sub-set of problems whose solution would have maximum effect. As used in this 
context, solution means an improvement in the conditions causing the problem. This improve- 
ment might require one or more research and development spirals. However, after each spiral, 
the new mix of problems was reviewed to determine if development should continue on the 
same problems or whether attention should be given to others. The result of using this approach 
means that no one element of the curriculum will probably be devloped to its maximum pot- 
ential. Rather, all or most elements will be developed to maximize the functioning of the 
curriculum. 

As mentioned previously, this research and developmcit, strategy can be utilized to evaluate 
and develop curriculum materials, research procedures, or teaching methods depending upon the 
needs of the moment. The procedure used to develop the curriculum materials is depicted on 
the spiral reproduced in Figure 4. Five steps are encompassed in this procedure. The first step 
is to define in as concise terms as possible the objectives for the segment of instruction to be 
developed. After this has been done, learning expedients can be procured or constructed to meet 
the objectives for the segment of instruction to be developed. After this has been done, learning 
expedients can be procured or constructed to meet the objectives. The next portion of the R & 
D cycle involves use of the expedients in an appropriate instructional setting. 

Data are collected prior to, during, and at the end of this tryout. The evaluation domains 
help define the areas in which to collect data and the criterion products selected will assist in 
the process of determining what ingredients and processes data to collect. 

The red bank around the cycle depicts the environment in which the steps are carried to 
completion. This environment interacts with these activities and may facilitate or hamper their 
accomplishments. For instance, if the tryout groups are comprised of high ability students, 
instructional materials will probably perform rather well. However, the same materials may fail 
to function well with students of lower ability. Thus, the results of the test must be interpreted 
in light of the environment in which they were obtained. The ingredients and processes domains 
supply the data for assessing the characteristics of this environment. 

A spiral may be broken into other activities for different phases of the development work. 
For example, in developing the evaluation system, the second step in Figure 4 would be replaced 
by the design of the evaluation system or sub-part of the sytem. 



Designing the System 

The evaluation system just described was evolved during the past three years. In fact, it 
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is still being refined and extended. This evolution was a planned part of the development sequ- 
ence for the system. Specific provisions for this process were built into the system and its rat- 
ionale. To date, this approach has worked effectively. 

Evolution was a necessary and logical procedure for the American Industry Project. At the 
start of the Project, the staff and the participating teachers had a limited amount of experience 
with research and development work. Moreover, there was a limited amount of time and money 
available for research and evaluation. Consequently circumstences-at least to some degree-pre- 
scribed this course of action. 

Gradual development provided the time and the opportunity the people in the system 
to grow with it. As a result, they were prepared to handle new concepts and procedures as 
these were put into effect. In-service training and experience with the evaluation system provided 
the basis for this controlled growth. 

Another factor that makes it necessary to evolve an evaluation system is the fact that it is 
difficult in the first stage of a project to envision all of the information that will be required 
and the operating characteristics of the evaluation procedures employed. An evaluation of data 
gathering technique can be developed only to limited degree before empirical evidence will be need- 
ed for its further refinement. This is especially true when a number of teachers with varied 
knowledge and attitudes are encompassed in the system. Initial experiences provide the informat- 
ion for the decisions that are required to continue the development of the evaluation system. 

Designing an evaluation system is not a simple task. Hence, the planner should utilize as 
many of the modern planning techniques as possible. The following, for example, have been 
instrumental in the development of cnt Project's evaluation. Network analysis has helped re- 
veal the sequence and contingencies in the activities required in the R & D work. Tree charts 
have been an aid in defining the consequences of a chain of decisions. Systems analysis techniques 
have played an important role in analyzing an organization which contains a number of elements 
working toward a common goal. Symbolic models have been used to assist the process of con- 
ceptualizing the evaluation system. 

Construction and use of symbolic models have been especially helpful in designing the eval- 
uating system. A model assists in reducing the information related to the resign of the system 
to a set of manageable and meaningful concepts set in a logical structure. In addition, thinking 
is aided by the symbols provided for the basic variables, concept”, and interactions identified in 
the model. These symbols can be manipulated mentally to deduce specific evaluation procedures 
and activities. 

The model building process is a fine blend of a knowledge of the final objectives for the 
evaluation system, the ability to generate ideas and ways to achieve these cbjec*'<es, and intuition. 
However, the process is self-supporting to a degree in that it challenges the clarity and validity of 
the objectives used in developing the model and stimulates thinking. 

One means utilized to stimulate ideas and fortify intuition has been ?Jv aj.-v-in-.ation 
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of Maria's work which relates retroduction and theory models to theory building education 
(Maria 1962) In brief, the procedure involves the use of an existing theory or law as the basis 
for a model which in turn provides a framework for a theory concerned with the area being 
studied. The source theory may be found in the same discipline as the problem or in other 
disciplines. 

The process of conceptualizing the environment that surrounds the cycles in the research 
and development strategy show in Figure 4 provides an illustration of the use of this approach. 
After selecting the spiral line to represent the R & D strategy, I proceeded to consider the de- 
tail of each spiral. Actions and their sequence were then derived without too much difficulty. 
However, at this point there was still a feeling that the spiral was incomplete in that it did not 
suggest how the ingredients and processes data were related to the activities on the spiral. Dur- 
ing the time this problem was being contemplated, the theory associated with conductors in mag- 
tic fields came to mind. As the ngredients, processes, and products variables were related to 
this theory, it became clear that there were many similarities. Intelligence of the experimental 
students, for example, could support or hinder the performance of instructional materials in the 
same manner that the direction of the magnetic field can facilitate or impede the flow of elect- 
rical current through a conductor. Thus, the environment component of the R & D strategy 
was developed with the electrical theory providing the model. 

Other source theories might have b?en just as appropriate. Ecology, for instance, could 
probably provide one or more source theories. However, my training made me more aware of 
the electrical theory and it proved to be of value. Note that this source theory "proved to be 
useful. There is no guarantaee that source theory will be appropriate. The resulting model or 
formulation must be tried out and evaluated. 

Effective communication is another problem to consider when designing the system. Part- 
icipating teachers, guidance counselors, students, and administrators will have some degree of invol 
vement in the evaluation. Incorrect responses or inaction may make the task of evaluating more 
difficult and time consuming and could mean the complete loss of some data. 



Systems analysis procedures have helped to define the components and interrelationships in 
the Project's evaluation system. Communications theory has been an important aid in developing 
communication within the system. Messages are constructed to fit the situation and th* receiver. 
Also, a procedures guide was written for the teachers, brochures on the evaluation have been 
distributed to the participating schools, and part of the Project's in-service workshop schedule has 
been devoted to discussing the evaluation in order to make the receivers more knowledgeable of 
the evaluation activities. In addition, care is taken to contact the appropriate people in terms of 
their function in the school and need for the information in question. 

Feedback from the communication network brings information for the evaluation and dis- 
closes the interference in the system. Hence, the person managing the evaluation must monitor 
the feedback and determine what changes need to be made. 

Managing the Evaluation System 

Responses to each communique, evaluation instrument, and procedure should be reviewed. 
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Initial responses to any new technique or procedure need to be analyzed very carefully. Embed- 
ded in this feedback will be the clues for possible revisions or information to substantiate the 
continued use of the original procedure. An example of this process is found in the review of 
a problem related to the distribution of information gathering forms to the participating teachers 
during the first year of the Project. These forms were to be administered by the te ers during 
the school year. The decisijn was made to give all of the forms to the teachers at the begin 
ning of the first semester. Sufficient quantities of each form were placed in labelled file folders 
arid given to the teachers at the end of the summer session workshop. 

The first form was administered during the initial week of school. All teachers returned 
these to the Project on time. The second form was to be administered periodically during the 
school term. Most teachers accomplished this; however, several "ran out of forms". The last 
form was to be completed by the students at the end of the term. Again, most teachers had 
done this correctly. However, the sheets returned by one teacher contained some copies of the 
Project's forms, copies of a somewhat similar form used by his school, and several sheets that 
did not relate at all to the information being collected. 

Actually, there was evidence of problems with this mass distribution procedure prior to the 
final returns. Earlier in the year I had occasion to visit each teacher. During these visits, it 
was observed that some of the teachers had not removed these forms from the box in which 
they received them. Moreover, in one case, the box was under several other items. The mass 
distribution procedure was based on the assumption that the teachers would file the forms. In 
practice this assumption was not met in all instances. The Project now mails the forms just 
prior to the time they are to be administered. 

Two important points are found in this illustration. First, it is worthwhile to go "where 
the action is". My visits to participating centers have been invaluable in determining the operat- 
ing characteristics of the participating teachers and the environment in which they work. Second, 
the effects of decisions have to be assessed. This review will indicate if changes should be made 
and it will also reveal some of the characteristics of the people influenced by the decisions. 

Continuous review of past decisions is an rntegra! part of the R & D strategy described 
previously. If this procedure is to be effective, initial and subsequent decisions must nrovide for 
flexibility in later actions. In the example given, there was little difficulty in changing the orig- 
inal decision. An alternative course of action was available and there were no psychological ram- 
ifications. This would not necessarily be true, however, of other decisions. Suppose, for example, 
that in the initial stage of the Project the teachers were assigned the rc^nonsibility for the com- 
plete evaluation of the American Industry courses without ?.ny intervention from the Project staff. 
How could this decision be changed without significantly disrupting the developmental and eval- 
uation activities? 

A basic rule in this instance is to not consider previous decisions, whether made by you or 
others, as proven facts. Error can be present in decisions as well as in data. T/ie very fact that 
decisions have to be made indicated that there are feasible alternatives and no certain course of 
action. Thus, it would be a good idea to keep a log of the various alternatives considered when 
making a decision and to indicate the degree of certainty associated with the one selected. 

This would be extremely helpful when the decision is reviewed at a later date. 



C-13 



117 



Decision makes must also be aware of the psychology that permeates the system. Response 
hierarchies and capabilities need to be determined. Rewards that effectively motivate actions 
should also be taken into consideration. Then this info nation has to be used in the decision 

making process. 

As evidence by the decision making procedures just described, efficient management of the 
evaluation system is dependent upon sufficient quantities of valid and reliable mformation. In all 
likelihood, the decision maker will have less data shen he desires, however, he must have some 
if his decisions are to improve the state of affairs. Lack of pertinent data adds more risk to the 
decision process. However, this lack should stimulate further development and refinement of the 
data gathering carrie 1 on in the evaluation. 



Summary 



In summary, the evaluation system utilized by the American Industry Project was designed 
to generate both evaluation data and information related to the management of the research and 
development activities. Experience with the system has demonstrated that management informa- 
tion is essential to the development work and the activities involved in collecting it are compat- 
ible with typical evaluation activities and helps to determine the factors associated with given 

outputs. 

Three domains, ingredients, processes, and products are encompassed in the Project's evaluat- 
ion. These domains provide the framework for evaluating the various learning materials produced. 
In addition, a cyclical research strategy is employed to provide for effective use of the data 
gathered in these domains. The feedback from an initial tryout of instructional materials provides 

a basis for further refinements. 

During the past three years, the Project's evaluation system has evolved into its present 
dimensions. The evaluation system was desired to provide for this growth so that the people 
in the system could mature with it. Also, past decisions and evaluation procedures are period- 
ically evaluated to determine whether thev are appropriate in light of the current status of the 
developmental work. 
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DEVELOPING AMERICAN INDUSTRY COURSES FOR 
THE SECONDARY SCHOOL 



Introduction 



The dual task of constructing a conceptual framework of industry in the United States and 
developing a completely new secondary curriculum based upon that framework might be described 
a» a monumental undertaking. The challenges implicit in this undertaking have been accepted by 
the American Industry Project and the description of this effort forms the substance of this paper. 
A teacher education program has also been designed and pilot tested for th : s new area of study. 
The material to be presented, however, considers only the conceptual framework and the develop- 
ment of the secondary program called American Industry.^ 

Major emphasis is placed upon: 

A. The need of the secondary school to aid all people in developing an understanding of 
American industry. 

B. The procedure used in identifying those knowledges necessary to understand American 
industry. 

C. The specification of behavioral objectives, taxonomically structured. 

D. The development of a logical course sequence. 

E. The development of instructional materials for teacher and student. 



Rationale 



The American Industry Project is in the process of developing a secondary school curriculum 
which focuses upon the knowledges necessary to understand American industry. With the forces 
that American industry exerts in the molding of our society, it must be recognized as basic to 
the American way of life. 

Technological advancements of industry affect our lives daily. New processes demand unique 
and more complex cognitive and nr -imputative skills. Labor saving products create new opportu- 
nities for thought and leisure. Automated machines using sophisticated electronic equipment result 
in efficient mass production. These changes are examples of how American industry interacts with 
society. It is also apparent that students are affected and must understand those forces which 
influence their lives. American industry has been identified as one of those forces and must be 
studied in order that students have an adequate understanding of their environment. Since 
American industry touches the lives of all people in our society, it is felt that American Industry 
courses are a part of general education. 



^ In this paper American industry refers to the enterprises found in the United States that 
exist for the purpose of making a monetary profit. American Industry refers to the courses 
being developed by the American Industry Project. 
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The American Industry program is centered about two broad objectives: 

A. To develop an understanding of those concepts which directly apply to industry. 

6. To develop the ability to solve problems related to industry. 

As a basis for meaningful curriculum development, it was necessary to identify a logical 
structure of generalized knowledges of industry. A conceptual structure for these knowledges was 
selected for four basic reasons: ^ 

A. Simplicity is provided, making details meaningful and more easily remembered. 

B. Conceptual organization provides a stable way of structuring knowledge. 

C. Relationships are revealed, which in turn provide a unity of knowledge. 

D. Conceptual understanding allows for transfer and the application of knowledge to 
various situations. 

The starting point for the development of the structure involved the formulation of a de- 
finition of American industry. Care was taken to use defining attributes which lent themselves 
to conceptual analysis. The Project formally defined American industry as: 

"an institution in our society which, intending to make a monetary profit, applied knowledge 
and utilized human and natural resources tc produce goods or services to meet the needs 
of man." 

A conceptual structure of the knowledges necessary to understand American industry was 
then developed. The structure consists of two sub-sets of concepts:^ (1) those concepts which 
apply to any industry regardless of product or service and (2) those concepts of our society 
that directly influence American industry in a manner that gives it a uniquely American character. 
(Figure 1) After this had been completed, conceptual models and definitions for each concept 
were developed.^ The production model is graphically presented in Figure 2 as an illustration. 
Production was defined as "those activities of an enterprise which are primarily concerned with 
the generation of specified goods or services." Each concept and sub-concept was defined sim- 
ilarly in terms of their respective attributes. This total structure was used as the basic reservoir 
of knowledge in developing American Industry curriculum materials. 



2 

Nancy Smith Nelson, "The Conceptual Approach to Teaching" (Madison: School of 

Home Economics, University of Wisconsin, 1964), Unpublished. 

3 

Concept is referred to as "a psychological construct resulting from a variety of experiences 
(detached from the many situations giving rise to it) fixed by a word or symbol, having functional 
value to the individual in his thinking and behavior." (Face and Flug, 1965, p.10) 

^Conceptual Structure, Models and Definition of American Industry, 1967. This contains the 
graphic representations of the basic concepts (Figure 1) and the definitions of each. It also 
contains the models (Ex. Figure 2) and definitions depicting the sub-concepts for each basic 
concept. 
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Fig. 1 Final American Industry Structure 
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'ig. 2 Production Model 



Development of Concepts, Models, and Definitions 



A first structure was developed by the American Industry Project staff, the Project's part- 
icipating teachers, and cooperating members of the faculty of Stout State University. The initial 
structure was viewed as a tool of inquiry, to be reshaped as insights were gained. Several cycles 
of investigation and refinement resulted in the conceptual framework shown in Figure 1. 

Following a review of available literature and some initial consultant contacts, consultant in- 
formation packets were prepared. These packets were designed to aid in fully developing and val- 
idating the concepts, concept definitions, and structure of American industry. They contained an 
introduction to the American Industry Project, guidelines for the consultants using them, and 
copies of the structure, models, and definitions previously developed. Over two hundred industri- 
alists, labor leaders, and educators reviewed these packets, each offering suggestions influencing the 
refinement of the original structure. 

A sample of the universities, organizations, and enterprises from which consultants were ob- 
tained would reveal representation of such institutions as: University of Minnesota, University of 

Chicago, University of Wisconsin, California Institute of Technology, United Auto Workers-Milwaukee, 
Carnation Company, American Federation of Labor-Congress of Industrial Organizations, North Amer- 
ican Aviation, American Marketing Association, Oscar Mayer and Company, and Minnesota Mining 
and Manufacturing Company. 

With the definition of American industry as a guide, the American Industry Project staff 
systematically analyzed the consultant responses and refined the conceptual structure, the concept 
models and definitions. This team effort by the staff utilized the system represented in Figure 3. 
Reflecting upon the definition of American industry, major concepts were first reviewed and revised 
where approrpiate. Then, starting with the definition of a specific concept, the definition was 
challenged to determine if it appeared to be logical and complete. If the consensus was that the 
definition was not appropriate, the concept was redefined. If the initial definition was found to 
be logical and complete, the next step included a review of additional information, at which time 
samples were collected of instances or examples that represented the concept area. These samples 
were reviewed to determine if the definition -subsumed all instances or examples. 



If the definition did not embrace all examples, the appropriateness of excluded examples 
was challenged. By challenging the anomalous examples it was determined whether to redefine 
the concept, or, if inappropriate, discard the examples. At this point, it was assumed that a 
logical and complete definition had been developed. The qualifications of the concept were then 
challenged. If the concept did not qualify as a major concept, the information was filed for 
inclusion in other concept areas and development of this concept stopped. However, if the con- 
cept did qualify as a major concept of American industry, the examples that were collected 
earlier were sorted into meaningful categories. Based upon the categories which had been identic 
fied, subconcepts were identified and defined. 
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major concept was challenged, the anomalous examples forced a redefinition of the sub-categories. 

It was then decided whether the sub-concepts qualified as an attribute of the major concept, if 
it did not, the information was recatagorized. If the sub-concept did qualify, it was included in 
the major concept. 

These techniques were used because they provided a method of organizing and analyzing our 
work. It is realized that other techniques may also fit with the development of conceptual 
course materials. However, by using the system described, th? models and definitions for each 
conceptual area were logically developed. Figure 1 gives the complete structure and identifies 
the major concepts that emerged from the study outlined above. The development of the total 
structure took about two and one half years. At the present time, the School of Applied Science 
and Technology, here at Stout State University, is continuing to analyze these substantive areas of 
American industry. 

A Taxonomical Breakdown of Level I American Industry 

Once the body of knowledge was defined, a taxonomical breakdown of the Level I Amer- 
ican Industry course was developed. 5 (See Figure 4) This breakdown was prepared by Orville 
Nelson and the author in an effort to structure a hierarchy of learning which in turn provided 
the basis for starting behavioral objectives. Two techniques developed by Gagne' were used. 

First, his appproach for mapping or structuring instructional content was used as a synthesis 
technique to obtain the logical sequence of the course. Second, this learning analysis procedure 
was used to define the elements to include in the course and to identify any hierarchies that 
existed in these elements. 

In developing any course, one of the basic considerations mus* be the educational objectives 
expected to be achieved, that is, the changes desired in students as a result of the educative 
process. For this course the hierarchy of educational objectives was developed in view of the 
concepts of American industry. Mager's (1962) work served as the guide to writing the objectives 
in behavioral terms. The sequence of these objectives may be seen by studying the taxonomical 
breakdown shown in Figure 4 and the course construction procedures described in the following 
paragraphs. 

Note Figure 4. A Taxonomical Breakdown of Level I American Industry, to obtain a graphic 
picture of the hierarchy of objectives. At the bottom of the page are listed the beginning 
competencies normally found with eighth grade students. Some basic competencies that can be 
expected are: eighth grade reading and math skills, normal psychomotor development, experience 



5 Level refers to a point of change in the course objectives in the American Industry cur- 
riculum. There are three levels of American Industry. Level I is the first year. Level II the 
second year, and Level III the third year. (Objectives for each level appear later in this paper) 
It should be noted that Levels I and >1 are being field tested on a semester basis as well as a 
full year. 
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in handling and spending money, and exposure to local enterprises. As in nearly any group, in- 
dividual competencies of students will vary, therefore, the teacher will have to compensate for 
these variations by using techniques and activities which are appropriate for his students and 
environment. 

This chart uses communication, one basic concept of American industry, as an example to 

g 

show how the Level I course is taxonomically structured. However, any of the concepts shown 
in Figure 1 might have been used as *he example. The first level of the hierarchical order in- 
volves simple recall of information or simple stimulus -response patterns. This could be merely 
naming specific facts or being able to give the names of various communication symbols without 
any real understanding of these facts or symbols. An example in the communication area would 
be simple recall of the senses used in communicating without understanding how the senses were 
involved in communicating or being able to say the words such as "sign", "person", and "behavior". 
The second level includes motor and verbal chaining. At his level the student could draw sym- 
bols used in communicating. He would also be able to link the names of symbols with the actual 
symbols. The purpose is to get the events in proper order. Lengths of the chains will vary with 
individuals. 

The next level invloves multiple discrimination . The individual working at this level would 
be able to distinguish among many events or stimuli in his environment. As an example, the 
person would be able to distinguish among the various communication symbols or acts. At the 
next higher level, the student would be able to group events or objects, classify them, and apply 
some generalizations to them. Sy responding to events, which he has mentally classified, he is 
actually applying conceptual reasoning. An example of applying conceptual reasoning would be: 
after observing many concrete situations which demonstrate the way ideas are conveyed from one 
person to another the student forms a concept of communication. If the concept is truly formed, 
the individual could then display a conceptual style of reasoning by applying his concept of com- 
munication to many other situations which are unique to him. This would be the test for his 
conceptualization of communication. 

At this level the student could also apply other concepts identified as basic concepts of 
industry. In this course, for example, he would be able to apply the concept of transportation 
by demonstrating how the movement of materials or people is planned and carried out. Or, he 
might apply his concept of research by solving problems through a systematic and unbiased in- 
vestigation of the problem. It should be noted that at this level the student would not inter- 
relate the concepts being applied. 

The next level deals with principle l earning . Principles are really chains of concepts, but 
with the relationships between concepts included. 8y breaking down concepts we can analyze 
the elements of each organizational structure and identify elements that form a concept and show 
the relationships between concepts. An oxample of principle could be shown in an analysis 



^Communication is defined as "the act of conveying ideas by means of signs and symbols 
for the purpose of affecting behavior." 
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of communication in industry. Let's assume that our concept of communication is basically, "the 
conveying of ideas by means of signs and symbols for the purpose of affecting behavioi." And, 
our concept of production is "the generation of goods or services." In analyzing any company 
we can form some simple principles by linking these concepts. 

A. In order to produce goods or services a description of the product is needed. 

B. Drawings and specifications may be used to describe objects. The principle being that 
communication is essential in order to produce goods or services. 

It is evident that to form principles, concepts must have already been formed. It is also 
apparent that hierarchical progress through the level of principle forming involves the linking of 
concepts. Therefore, complexity and sophistication of the princip' • grimed are directly related to 
conceptual understanding. 



Problem solving is at the upper level of the taxonomy. Realizing that an infinite variety 
of problems exist and that problems are solved at many levels of difficulty, it is still possible 
to isolate some common characteristics of problem solving. Problem solving involves: 

A. Developing a plan of procedure for solving the problem. 

B. Exercising judgments in determining what information is required in solving the problem. 

C. The arranging of parts or information to form a logical pattern. 

D. Discovery of new information .7 

The term "problem" is directly related to the solver. What might be a problem for one 
person could well be "old hat" for another. In essence, the concepts and principles formed by 
the learner will be a determiner of the problems he can solve and the level of his answers. 

Bloom's Taxonomy of Educational Objectives (Cognitive Domain,l965) also served as a quide 
in preparing objectives for this course. The taxonomical breakdown by Bloom combined with 
Gagne's work provided the base for a meaningful classification system for the course objectives. 

The psychomotor domain was also considered in developing this course. Specific motor 
skills are taught during the first three units of the course However, this does not include the 
development of a specific group of skills for each student, but skills needed by an individual in 
order to carry out classroom activities. That is, if the student activity consists of mass producing 
wooden coasters which requires the use of the band saw and disc Sander, not all students who 
need to know (operators and foremen) would be taught these skills. 

Some general manipulative skills would be taught, however, the focal point would be the 
concept of skills learning. It would involve the process of learning a skill, and not necessarily 
how to operate a particular piece of equipment. The learning would be conceptually oriented 
in order that the information and process could be transferable to other learning situations. 

This allows the students to learn a variety of manipulative skills with greater efficiency through 
the use of a conceptual orientation to skills learning 

The affective domain is reflected in the course materials and student attitudinal changes 
are brought about in various ways. An example is safety. In the American Industry classes 
safe use of machines and tools is demonstrated, safety literature is distributed, and safe practices 
in industry are noted and posted. Some other attitude changes expected are developed in the 
students' feelings toward profit, competition, government, and industry. 
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Fig. 5. THE DEVELOPMENT OF LEVEL l CVRHICI LLM MATERIALS 



Having discussed the method and rationale involved in preparing the educational object' 
ives of this course, an overview of all American Industry curriculum materials is in order. 

The Deve’opment of Level I Curriculum Materials 



With the structured body of knowledge as the basic source of information and the 
taxonomical breakdown of the course objectives in mind, let's view the development of 
the Level I curriculum materials. (Sec Figure 5 - steps in developing the first level mater- 
ials. 



Development started with preparing the rationale for instituting a new curriculum en- 
titled American Industry (Face and Flug, 1965} The rationale contains the fundamental 
reasons for a new curriculum and the general objectives of the curriculum. The general 
objective of ♦he American Industry courses were seen as: 

A. To develope an understanding of those concepts which directly apple to industry. 

B. To develop the ability to solve problems related to industry. All materials for 
the American Industry courses were developed with these two objectives as a base. 

Next, objectives for the ihree levels of American Industry were identified. They 

are: 

LfiVSl 1 , .designed for eighth grade students, has the following course objectives: 

1. To develope 8 knowledge and understanding of the major concepts of industry 
and their relationships. 

2. To develop the ability to solve simple problems related to industry. 

Level 1 1 . designed for tenth grade students, has th<* following course objectives: 

1. To develop in-depth understanding of the concepts of industry, and develop refined 

understandings of the relationships among the concepts. " 

2. To expand the ability to recognize and solve complex problems related to industry 

Level III, which is for twelfth grade students, id designed to develop knowledges and 
pro Nam solving skills within a concept area or cluster of concept areas aDprpriate to the 
individual's level of ability and interests. 

Next, the course outlines were prepared in accordance with the objectives (general and 
i.ivel) and the conceptural method of presentation , 

From this point forward, this paper will detail the preparation of Level I materials. 
Levels II and III are being developed in a similar manner. 



Level I, Unit Outline and Objectives 



lr describing the unit outline for Level I, it should be noted that unit objectives 
were alsc identified and later were followed by lesson objectives. Each set of objectives 
was built upon the basic objectives. The sequence in preparing objectives was as follows: 

A. General American Industry objective:. 

B. Level objwCtives 

C. Unit objectives 

D. Lesson objectives 

Figure 6, Level i - American Industry Course Outline was prepared with the aid of 
various structuring methods. The seven units outlined represent: 
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Unit and Features 



Unit Theme 



TEACHER DIE: TED 



I ^et's analyze 
industry. 



The needs of 
man and why he 
progressed. 



Let's start a 

business. 




Why does a pro- 
duct sell? 



Where do we go 
from here? 



- The students 
organize, pro- 
duce , and sell. 



STUDENT DIRECTED 



Fig. 6 Level l - American Industry Course Outline 
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A. A course that utilizes the cyclical approach and is designed to "cycle" through 

the basic concepts of American Industry once in Unit I and once in Unit II. 
Individual concepts are studied in greater detail during Units III, IV, and V. At 

the completion of these three units the student will have studied each of the 
concepts and thus will have completed another cycle. Each time a cycle is com- 

pleted the student progresses to a more sophisticated level of understanding as is 
represented in Figure 4. At the higher levels the student is problem solving at 

a level commensurate with his abilities. 

B. Another structuring method involves moving from a teacher directed activity to 

student d ire cted activity. That is, when starting the course most activity is teacher 
directed, however, as the course progresses activity direction is gradually shifted to 
the students. This allows for in-depth problem solving which is necessary to 

reach the higher levels of learning. 

After the Level I objectives and outline hed been completed the following activities 

were started: 

A. A Level I film catalog was prepared. This catalog contains reviews of 16mm 

sound films suitable for Level I students. It is designed to assist teachers in 

locating films that directly relate to the conceptual areas taught in American In- 
dustry, therefore, the films are grouped according to concept area. Each film has 
been previewed and found to be acceptable for the stated level. The title, rental 
arrangements, cost, company, and a brief synopsis is presented for most films. A 
total of 141 films is listed in the Level I catalog. (Daehling and Wolf, 1967) 

It should be noted that, although the films previewed sre the best of those lo- 
cated, future plans include development of conceptually oriented films more directly 
related to the course materials. 

B. The lessons were also started when unit outlines and objectives were complete. In 

developing the lessons the following process was followed. First, themes were in- 
troduces as variants of the concepts to be studied. The general unit objectives 

were then reviewed. Next, the detailed objectives for the lessons were developed^ 

At the same time, however, lesson length was established. When the objectives 
and outline for the entire unit had been completed, the lessons were written. 

Each lesson consists of the following: 

1. Theme (variation on the concept or principle involved). 

2. Behavioral objectives which were developed according to the procedure 

defined in Figure 4, A Taxonomical Breakdown of Level I American 
Industry. 

3. A general description, cr scope, of the lesson. 

4. Resource references identifying instructional media, and literature for 

students and teacher. 

5. Lesson content which has been written in dialogue form addressed to 

the instructor. Lesson content consists of an introduction, activity and 

co n tent , and a conclusion. In the lesson content, the resource reading, 

lesson activities, and instructional media have been appropriately keyed 
into that lesson. 

6. Student activities are also a part of these lessons but will be discussed 

later. 

7. Evaluation items are included in each lesson, however, it has been 
suggested that the teacher use these only as a guide to preparing his 
own evaluation. 
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C. The student activities were designed by the same person who prepared the lesson 

and became a part of the lesson. In many lessons there are sufficient activities 
to allow a selection from two or three activities available for that lesson. These 
activities allow the student to become involved in rea[ industrially related learning 
situations. Examples of some activities used in Level I include: planning for 

and mass producing wooden coasters and silk screened pennants, going on field 
trips to observe natural resources or to visit an enterprise, posing labor-management 
problems and solving them through arbitration, establishing a small student enter- 
prise, selling stock in the company, doing research on a product, performing mar- 
ket research, planning for continuous production of a product or a service, pre- 
paring financial and personnel records, conducting hiring interviews, and writing 
final reports indicating profit or loss of the company. 

This represents a cross section of the activities included in the Level I instructor $ 
guide. Later, there will be a further explanation stating how the entire instruc- 

tor's guide operates. 

D. Project developed media. As each lesson and its activities were completed the 

media for that lesson was started. A film, overhead transparency packets, slide 
series, audio tapes, graphs, charts, mobiles, and bulletin board displays have been 
developed for use in this course. As mentioned before, each has been keyed to 
a specific lesson to allow the teacher to quickly identify its location in the 

lesson. 

E. Student texts were written for each of the seven units in Level I. Written in a 
contemporary style, the student texts contain examples of simple industrial events, 
the principles of which are readily transferable to situations in various companies. 

Each test contains a table of contents, illustrations, bibliography, and definitions of 
the salient terms used. Text reading assignments are also keyed to specific lessons 
in the instructor's guide. The reading level of the student texts has been eval- 
uated. Determination of the reading level was done by Mrs. Sylvia Nelson, a 
person with a background in reading research. In establishing the reading level 
Mrs. Nelson used the Dale-Chall (1948) formula. 

General Operating Characteristics of the Course 

The following provides an overview of the methods of operation employed in course 

development. Details of the procedures are found in the "American Industry Procedures 
Book" (1967). 

Using the Instructor's Guide and Student Texts 

Before attempting to teach American Industry courses, an understanding of the philosophy 
and rationale is necessary. It is also necessary to know how the instructor's guide is used 
and how to safely demonstrate equipment used in the laboratory. 

As mentioned before, the guide is divided into units and lessons. In each unit, the 
first lesson provides the introduction. Successive lessons develop the concepts of American 
industry through the cyclical method . In each cycle, the aim is toward deeper understanding 
of the concepts. In Units III, IV, and V the basic concepts are the first studied individ- 
ually, and then the relationship between the concept being studied and those previously studied 
are discussed. 
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In using the guide and student texts, the teacher should start by perusing the intro- 
duction and course outline. Next, the units, corresponding student activities, and student 
text materials should be studied. At this point the teacher should have evol d an over- 
view of the course about to be taught. He should then read the student text materials 
and mentally interweare the lessons and student activities with the student texts. 

When the teache' has accomplished the above, he should study each unit, lesson by 
lesson, and detail his course in prepa ation for teaching it. This should include the ident- 

ification of selected activities and approximate lesson dates. 

As in any new course, it is necessary to determine the effectiveness of the course 

material. For the American Industry Project a system was designed to gather feedback in- 
formation from teachers and students using the course materials. Many of the revisions came 

as a result of student and teacher feedback forms. In addition, classroom observation, teacher 
interviews, and student contacts provided feedback data. 



Feedback and Revision of Curriculum Materials 



As mentioned, improvement of these materials has been accomplished through many 
means. Observation in the classrooms and laboratories, followed by interviews with partici- 
pating teachers and students has provided much data. The Project's supervisor of participating 
teachers visited each school monthly. During his visits to participating centers he recorded 
information pertaining to the instructional technique being used, instructional nadia employed, 
and the students reaction to the presentation or activity. The visitation record prepared by 
the supervisor of participating teachers is discussed with the participating teacher following 
each observation. Additional suggestions for improving curriculum materials are often made 

during these discussions. 

Another feedback tool is tha lesson feedback form. One feedback form is completed 
by the teacher at the end of each lesson. It contains an overall rating of the lesson , an 
appraisal of the objectives, content, instructional media, plu; other general comments. A 
similar form is prepared as an overall rating of each unit . 

A media feedback form is completed each time instructional media are used by the 
participating teacher. This form also provides data necessary for media revision. 

The student reading material is rated by the students and the teacher. Each student 
is encouraged to make comments within the booklets and underline words he doesn't under- 
stand. Students are also asked to note any lack of continuity they feel exists. Finally, 
the students complete a formal rating of each booklet, the Nelson Rating Scale for Written 
Materials. This form is designed to gather a sequence of information related to the r *» r.*< 

attitude toward the booklets, vocabulary, the style and layout of each booklet, re* "*« 

the booklets, and content. 

Teachers also examine each booklet and note spelling, punctuation, and V errc i. 

In addition, they check literary continuity and suggest addition or deletion of .<al. 

All feedback infor mation is recorded and filed by unit and lesson to be used for 
revision purposes. In making revisions each comment is reviewed and analyzed before 

making the change. 
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iluation of Curriculum Materials 



In cider to determine the effects and operating chai icteris v.s of the American Industry 
courses, evaluation procedures have been developed and employed. A wid*; v>actrum of evaluation 
procedures was designed to obtain descriptive and comparative data. Details of the evaluation 
may be found in a paper by Orville W. Nelson (1967). The paper is available from the Amer- 
ican Industry Project. 



Conclusion 



The previous pages have described the procedures, rationale, and philosoji .y which fund- 
amental to the American Industry Project. A logical structuring of conceptual models, combined 
with taxonomically arranged objectives provide a hierarchical learning pattern for students of 
American Industry. 

The instructor's guide serves as the focal point for all course materials. That is, it provides 
guidance for the teacher in identifying a consistent course pattern; it identifies supplemental 
reading for teacher and student; it provides suggested student activities, it combines suitable 
mjdia with prescribed lessons; and it outlines evaluation items. By combining the instructor's 
guide, student texts, and instructional media one can envision an instructional package designed 
for teaching the concepts of American Industry. 



REFERE NCES 



American Industry Project Staff. Conceptual structure, models and definitions of american in- 
dustry. American Industry Project, Stout State University, Menomonie, Wiscons a, Feb- 
ruary, 1967. 

Bloom, B. 3. Taxonomy of educational objectives - handbook I cognitive domain. New York: 
David McKay Company, Inc., 1956. 

Daehiing, W. and R. Wolf. Suggested films for teaching tirst level american industry. American 
Irdustry Project, Stout State University, Menomonie, Wisconsin, 1967. 

Dale, E. and Jeanne S. Chall. A formula for predicting readability: instructions. Educational 

Resource Bulletin. 1949. 

Face, W. L. and E. R. F. Flug. The establishment of american industry as a transit* .riel subject 
between general and vocational education. American Industry Proposal to USOE, Sunt 
State University, Menomonie, Wisconsin, 1965, 



D-18 



1C*9 



Face, W. L. E. R. F. Flug, W. Daehling, R. Gebhart, O. Nelson, H. Olstad, and J. Zuerlein. 
American Industry Procedures Book. American Industry Project, Stout State University, 
Menomonie, Wisconsin, July, 1967. 

Gagna', R. N. The conditions of learning . New York: Holt, Rinehart and Winston, Inc. 1965. 

Mager, R. F. Preparing instructional objectives. Palo Alto, California: Fearon Publishers, 1962. 

Nelson, Nancy Smith, The conceptual approach to teaching. School of Hsrne Economics, Univ- 
ersity of Wisconsin, Madison, Wisconsin, 1964. 

Nelson, O. W. The american industry project: development and evaluation, prepared for curr- 

iculum programs, in action: their administration and evaluation. American Industry Project, 
Stout State University, Menomonie, Wisconsin, February, 1967. 



D -19 




139 



Appendix E 



Modern Curriculums Demand 
New Media and Methodology 



by 



John Zuerlein 

Instructional Media Specialist 
American Industry Project 
Stout State University 
Menomonie, Wisconsin 



May, 1969 



American Industry Project 
USOE Contract No. OE-5-85-060 
Co-Directors: 

Dr. Wesley L. Face 
Dr. Eugene R. F. Flug 



E-i 



140 



"Modern Curriculums Demand New Median and Methodology" these words formulated the 
direction and philosophy in the development oV instructional media for the American Industry 
Curriculum. Unlike many present industrial arts curriculums, American Industry does, in fact, 
require new and modern media which, in turn, must provide for new methodology in utilization 
as .veil as in student and teacher response. 

An attempt was made to make the media "come alive" and to HAmand the integration of cur* 
riculum materials, concepts, and knowledges. In order to do this, all media were directly cor- 
related with the lessons and their objectives. They were designed to stimulate discussion rather 
than merely support lectures and present facts which would be regurgitated later by the some- 
times familiar student memorization robot. 

The "o!an it attack", so to speak, followed, with some modification, a twelve point develop- 
ments procedure established by Dr. David P. Barnard, Director of Stout State University's 
Audio Visual Center. These twelve points are as follows: 



1. Delimit the subject matter area. 

2. Identify behavioral objectives. 

3. Identify the curriculum structure. 

4. Identify the course content. 

5. Plan the program budget. 

6. Select the curriculum specialist. 

7. Form a curriculum specialist media specialist relationship. 

8. Plan the media. 

9. Produce the media. 

10. Check the media 

11. Utilize the media effectively in teaching. 

12. Evaluate the media 



It is quite evident that this directive was most conducive to a team approach. Such an ap- 
proach was adopted, and the media development began. 

It would be advantageous, here, to trace the development of the media to see just how it was 
correlated within the curriculum. 




Delimiting *he Subject Matter Area 



The subject matter for the American Industry curriculum, initially established by Dr. Wesley 
Face and Dr. Eugene Flug, and later by Richard Gebhart was the foundation upon which all 
materials were to be developed. The decision to study “those concepts directly related to in- 
dustry'' opened the doors for the development of supporting mediative presentations and materials. 
It also supplied the directive for various other curriculum materials for which media were to 
lend support. 

Identifying the Behavioral Objectives 



The two overriding behavioral objectives established by the Project were: 

1. To develop an understanding of those concepts which directly apply to industry. 

2. To develop the ability to solve problems directly related to industry. 

Specific unit and lesson objectives ware developed later. All of these objectives, then, became the 
overriding objectives of the media. 

An attempt to reach an understanding of “those concepts which directly apply to industry" 
through mediative presentations was evident in the media content, which demanded discussion. 
Understanding through discussion, then, was to lead to the solution of problems. Many of these 
problems were included in the content of the media. 

Identifying the Curriculum Structure 

From the preliminary framework evolved the present American Industry structure. Figure 
1 shows this structure. 

Identifying the Course Content 



The structure provided a guide to the development of the curriculum materials, which were 
to include an instructor's guide, student booklets, and instructional media. The content was de- 
rived from the knowledges of industry related to the structure shown in Figure 1. 

Plan the Program Budget 

Budgetary items involved with media development fell under the headings of: 

1. Laboratory equipment 

2. Production supplies 

3. Staff 
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Fig. I. FINAL AMERICAN INDUSTRY STRUCTURE 
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Laboratory equipment included such items as light tables, audio* visual projection equipment, 
drawing and graphics equipment, etc. The production supplies took *he form of papers, acetates, 
paints, and various other expendable items. As for staff, one full time staff member designated 
as "Instructional Media Specialist" was named. One graduate assistant and one work study stud- 
ent were also assigned to this area. 

Select the. Curriculum Specialist 



A curriculum specialist was designated to write the curriculum materials for American Indus- 
try. Several participating teachers were also selected to field test the total program. 

Form the Curriculum Specialist. Media Specialist Relationship 

Figure 2 shows the relationship between the Curriculum Specialist and the Media Specialist. 

It was the task of the Curriculum Specialist to develop the specific lesson objectives and to draft 
a preliminary lesson plan. Once this was completed, the lesson and objectives were given to the 
Media Specialist for an analysis of possible media that could enhance the lesson and assist in 
reaching the objectives. 

The type of media, if any, was to be selected. This was the task of the Media Specialist. 
Since the various types of media (filmstrips, slides, overheads, instructional films, etc.) lend them- 
selves to certain situations more effectively than others, a decision of which to use was necessary. 
Also, participating teachers had more overhead projectors available than slide projectors. Thus, 
overhead transparencies were selected more frequently than slides as the media. 

Once this decision was made, the content of the media was determined. This was general- 
ly dictated by the lesson content and objectives. Here, the Media Specialist and Curriculum Spec- 
ialist worked closely together to insure the proper presentation of information so as to most 
effectively reach the established objectives. 

With the content stabilized, the media was written into the lesson. Here again, the media 
and curriculum specialists jointly decided on the methods of presenting the media within the 
le&on plan. 

At this point, the actual media was to be developed. The process for developing the 
media follows: 

Plan the Media 

The first step was to review the lesson plan. Realizing that excellent and efficient instruc- 
tion is necessary in meeting the objectives, planning of the media and mediative presentations, 
was of utmost importance. 

information pertaining to the media was placed on "Instructional Media Job Sheets". 

Figure 3 is an example of a job sheet. This sheet listed all information necessary for planning 
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Develop Objectives 






CURRICULUM SPECIALIST - MEDIA SPECIALIST 



Date 



Job No. 



Com filet ion Date 



Level No. 

Unit No. 

Lemon No. 

Activity No. 

Tentative Title 

Objective to be met: 



Suggested Design: 
Medium: 



(Attach to this sheet the content outline and complete activity sheet.) 



Fig. 3. INSTRUCTIONAL MEDIA JOB SHEET 
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PRODUCTION TITLE 

DATE 

MEDIUM— 16MM Q TV □ FILM STRIP Q 2 x 2 TV SLIDES □ 
COLORS WANTED ON CEILS 

FRAME NO NARRATION 




FRAME NO 




Fig. 4 AV PLANNING SHEET 
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SHEET NO 

JOB NO 

2x2 SLIDES r 
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the media and included any suggestions for design and medium (overhead, slides, etc.). Prelim- 
inary sketches were made. These preliminary sketches included graphs, diagrams, illustrations, 
and lettering. Preliminary narrations were written for filmstrips and slide series. 

Audio Visual Planning Sheets were used for preliminary planning of the filmstrips and slide 
series. An example is shown in Figure 4. The AV Planning Sheet has space for both narrations 
and sketches. 

Student interest war another major concern in the planning stages of the media. Four 
major items were considered in designing the media to hold student interest. Color was one 
aspect, and lots of it. Color was prevalent in all slide series and filmstrips. An array of anal- 
ogous, complementary, triad, and monochromatic effects were selected for the various frames. 

Since color was pre-determined when using diazo film, illustrations had to hold the interest. 
Single line drawings and cartoon characters supplied needed interest factors for overhead trans- 
parencies. while illustrations in the slide series and filmstrip often took abstract forms. Examples 
of each may be seen in Figure 5. 

Narrations were of prime importance and presented some unique problems. Quite often, 
as in the use of some slide series and all overhead transparencies, the instructor had to be the 
narrator. Scripts were supplied for all slide series. Activity and presentation-discussion guides 
were written into the lesson plans of tie instructor's guide for use of the overhead transparencies. 

The narration for the filmstrip "The Future of Industry" was recorded on audio tape, and 
synchronized with music, music being the fourth interest factor. 

All four interest factors were not present in every visual. However, at least one, and 
quite often two, of these factors was present in each and every complete visual. 

Produce the Media 

Overhead Transparencies 



A total of 57 overhead transparencies were designed and produced fo.* the first level cur- 
riculum materials. After preliminary sketches were made, finished artwork was produced on 
tracing vellum. A combination of india ink, rub on letters and figures made up the overhead 
masters. Transparencies for all 13 concept areas were designed and included in the instructors 
media packet. The Diazo process was used in the production of the actual transparencies. 

In addition to curriculum materials, staff presentation overheads were produced. These 
overhead transparencies were utilized in various small group presentations, convention presentations, 
and educational institution presentations. All transparencies were developed in the American 
Industry Media Lab. Additional facilities which were not available from the American Industry 
Media Lab were supplied by the Stout Audio Visual Center. 
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Fig. 5 SAMPLE ILLUSTRATIONS UTILIZED IN 
THE AMERICAN INDUSTRY INSTRUCTIONAL MEDIA 
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Slides and Slide Series 



A total of five slide series were developed for the first level curriculum. A brief description 
of each follows: 

1. "What Does the Government Do For You." This series contains eighteen slides. It 
discusses the various governmental regulations placed upon our U.S. economy for public 
protection. Items covered in the series include our judicial system, interest rates, com- 
munication controls, weather forecasting, insecticide specifications, food and drug adm- 
instrations, transportation, pollution, and personal benefits. This series is furnished 
with a written narration. 

Evaluation (covered later) has indicated that this series did not meet the course 
objectives. It was from the curriculum materials. 

2. "A Capitalistic Economy". Forty-five frames comprise this series which also contains 

a written narration. The illustrations took the form of cartoon characters. The series 
deals with the development of capitalism and its status in the American economy. A 
brief history of its development is included. 

The aspects of private property, free enterprise, individual freedom, incentives, 
and profit are discussed as the basic features of the American capitalistic economy. 

Again, evaluation dictated the removal of this slide series. 

3. "Man to Machine." "Man to Machine", a slide series containing 31 slides, traces the 
development of machinery in the textiles, farming, drafting, and construction industries. 
Some of the earlier machines are introduced as well as those presently being used in 
these industries. This series is illustrated by actual photographs and drawings and does 
not utilize abstract symbolism. 

4. "An Introduction to Production". The last slide series utilized in Level I American 
Industry is entitled "An Introduction to Production ". It contains e. total of 62 
slides and is designed to help the student better understand production as it relates 
to industry. Various aspects of production, such as scheduling, job assignments, qual- 
ity control, processes, industrial relations, facilities, materials and materials handling, and 
planning are visualized and discussed within this series. Photographs of actual indust- 
tial processes, charts, graphs, and plant layouts are included in the illustrations. 

6. "Advertising is an Invitation". The latest slide series to be introduced to the curric- 
ulum is entitled "Advertising is an Invitation." This series contains 70 frames. It is 
a self-contained teaching unit that may be viewed individually or in a group. The 
narration is included on each frame. A built in test is also included. 
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The v?nous a?p«f .« of advertising media and their appeals are discussed. Several examp. j& 
of each are included. The built in test covers each phase of the series by asking the student to 
identify appeals, select effective advertising media for various products, relate advertising to the 
communication process, and to design an actual ad. This series has not yet been evaluated or 
utilized in the classroom. Its release is pending approval of several corporations whose products 
and trademarks are displayed within the series. 

Several hundred slides were also developed for staff presentations. These slides were catal- 
oged and placed in a reference file for staff members to check in and out. 

Film Strip : "The Future of Industry" One flimstrip containing 36 frames was designed 
for Level I American Industry. Two facilities were used in the production of this filmstrip: The 
American Industry media lab, and commercial photographic facilities. All artwork and narration 
were complete in the American Industry and Stout Audio Visual Laboratories. The process 
used for illustrations (which took very abstract form) was as follows: 

1. A detailed working drawing was completed in pencil, within a specified format. 

2. Prepared acetate was placed over the drawing and the illustration was then traced 
onto the acetate in India ink. 

3. Cartoon paint was then applied to the reverse side of the acetate. By doing this, 
the sharp outlines (made in ink) could be maintained. The colors, which dry in a 
matte finish, were given added brilliance when viewed through the aceta'a. 

4. All backgrounds were given a soft smooth appearance by utilizing various airbrush 
patterns on poster board. 

The narration and background music were taped in the Stout Audio Visual Sound Lab and 
synchronized onto one audio tape. Ail artwork was then set to a commercial photographer for 
photographing, packaging and labeling. 

The purpose of this filmstrip is to present a number of possible ideas that may become a 
reality in the future and how theyn might affect our society. It is designed to meet the follow- 
ing objectives: 

At ths conclusion of this lesson the student should be able to: 

1. Reflect upon the accomplishments of man and logically predict some of man's future 
accomplishments. 

2. Indicate areas of industry that are expected to expand through the development of 
new products. 
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16 MM F ilms. One 16mm film entitled "American Industry" was produced by the Stout 
Audio Visual Center, in close cooperation and consultation with the American Industry Project 
staff. This film is 19 minutes long with sound and color. Its purpose is to provide the viaw- 
ers with a basic conceptual understanding of American Industry. Students will become familiar 
with the elements of industry, the typical sequence of industrial activities, and the similarities 
among industires. In addition, the film tells how industry affects the student. The film stresses 
these six points: 



1. American Industry affects everyone in our society; and, therefore, we should have an 
understanding of it. 

2. Industries arc similar in many ways and the basic similarities are identified as: (a) 
products are needed, planned, produced, and sold to attain a profit and (b) in order 
to produce products industry needs men, machines, an d materials. 

3. A combination of men and the extension of his capabilities, machines, are related to 
output efficiency. 

4. Communication plays a major role in industry; whether it be between man and man 
or man and machine. 

5. Testing and quality control insure the production of a product that meets desired 
standards. 

6. Company sales rely upon the ability to persuade people that their product best meets 
the customer's needs. 

Film Catalogs . Three film catalogs and two supplements were developed for classroom use. 

All films were previewed to determine the grade level at which each film should be viewed. 

This was determined by the level of understanding typical of each grade. 

Over 5,000 films were previewed, evaluated, and catagorized according to concept area. 

Each film catalog lists films alphabetically in each of nineteen areas: Introductory Unit, Com- 

munication, Transportation, Finance, Property, Research, Procurement, Relationships, Marketing, 
Management, Production, Materials, Processes, Energy, Resources, Competition, Public Interest, 
Culminating Unit, and History of Industry. Each catalog is also designed to assist the instructor 
in finding films that directly relate to the areas taught in the American Industry Curriculum. 

One catalog and a supplement for use in Level I, an additional catalog and supplement for 
Level II; and one catalog for Level III were developed. 

Overhead Transparency Masters. A jf overhead transparency masters have been produced. 
These were printed on mimeo bond paper and were intended for use in a thermofax or 3M 
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copier. Each booklet contains masters for all overhead transparencies used in the Level I Amer- 
ican Industry course. They have been arranged by a numbering system, coded for assembly and 
filing purposes. Figure 6 shows the numbering system. These booklets contain a complete 
description of the code and assembly instructions for each type of overlay, plus additional tips 
on adding cn'or and giving visual presentations. 

Print— i M aorieh. The Instructional Media Laboratory and staff assumed the responsibility 
of duplicating most of the printed publications concerning the American Industry Project. Of 
the many and varied materials reproduced, the seemingly most significant are listed below: 

1. Level I, II & III Instructor's Guides. 

2. Descriptive pamphlets for mailing packets (13 different pamphlets). 

3. Overhead Transparency Masters Booklets. 



Level 




Fig. 6. NUMBERING SYSTEM FOR OVERHEAD TRANSPARENCIES 

4. Evaluation Materials 

5. Evaluation and Informative Reports 

6. Two newsletters. 

In addition tc the above mentioned, various letters, announcements, covers, program desc- 
riptions, proposals, feedback information, and post cards were reproduced. An estimated 20,000 
reams, or 10,000,000 sheets of paper have been processed through the Gestetner reproduction 
machine. 

Check the Media 



All materials must be checked. Instructional media is no exception. Nothing is more dis- 
tracting than a misspelled word; a backwards slide; an unpleasant color scheme; or false information. 
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As soon as the media was complete, it was previewed and checked against the objectives. 
All necessary changes, if any, were made at this time. 

Utilize the Media Effectively in Teaching. 



All media must be utilized in order to determine its effectiveness. The American Industry 
media was introduced to the participating teachers at area workshops. It was the purpo* here 
to instruct the instructors on the intent and use of all media. Various approaches to use and 

to time of use were presented at these workshop,. Follow up activities were also suggested. 

With this background, then, the teachers were ready to utilize the media in actual teaching 

situations. Each instructor was issued one complete set of media. The availability of media was 
deemed necessary. Far too many good instructional devices have been overlooked due to inacces- 
sibility. This was not the trend in the American Industry Curriculum. The problems, if any, 
were in terms of inaccessibility of projection equipment rather than media. 

Evaluate the Media 

Evaluation of the media was conducted by various feedback sheets which were filled out by 
the instructors. Figure 7 indicates the response totals from instructional media information sheets. 
By studying the responses in Table 1, an overall picture of the effectiveness of the media in each 
unit can be obtained. Problems can also be detected, such as the inaccessibility of projection 
equipment. The large numbers in the "no response" column towards the latter units can be at- 
tributed to the fact that many instructors were behind schedule in planning and neglected to re- 
turn the information. 

By and large, the overhead trasnparencies were used by ail instructors. They were reported 
to be in excellent physical condition and most instructors had access to an overhead projector. 

The overall effectiveness was rated high except for Unit I. Feedback indicated the needed infor- 
mation to enable these transparencies to be revised. 

All slide series and filmstrips were reported to be in excellent physical condition. The ac- 
cessibility to remote controlled projectors was almost nil); however, most schools had some type 
of projection equipment available. 

The slide series "What Does the Government Do Fur You" and "Capitalistic Economy 
which appear in Unit I received much criticism and only average rating in terms of effectiveness 
in reaching objectives. After due consideration, it was decided to drop these series from the 
list of media and to delete them from the Curriculum Materials. The remaining slides and film- 
strip received high ratings, with particular praise for ihe filmstrip "The Future of Industry. 

Lesson and unit feedback and questionnaires to participating teachers also supplied evaluation 
o' *he media. Table 2 shows these questions and responses along with several interesting facts, 

tt.^se being; 
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Instructional Media Information H espouses 



SECTION l • Overhead Trasnpareneies 

l. Indicate your usage of the overhead transparencies furnished for this unit. 

Unit I , Ul I <> :{ 

AM SOME W£!i M) response 

Unit I Id I 0 :i 



Unit II 
Unit III 
Unit IV 
Unit V 
Unit VI 
Unit VII 



None available at time of evaluation 

9 I 0 7 

() 2 0 (> 

6 1 0 9 

None availabl ? at time of evaluation 

2 0 2 n 



Indicate the physical condition of the transparencies. 





EXCELLENT 


GOOD 


FAIR 


POOR 


NO RESPONSE 


Unit 1 


13 


1 


0 


0 


3 


Unit II 


None available 


at time 


of evaluation 






Unit III 


10 


0 


0 


0 


7 


Unit IV 


10 


0 


0 


0 


6 


Unit V 


7 


0 


0 


0 


10 


Unit V! 


Non'' available 


at time 


of evaluation 






Unit VII 


2 


0 


0 


0 


ir> 
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7/1 HI, I 1 ' I continued 



indicate your accessibility to an overhead projector. 



Unit I 
Unit II 
Unit III 
Unit IV 
Unit V 
Unit VI 
Unit VII 



ALWAYS 

Available 



SOMETIMES 

Not 

Available 



IHEEICULT IMimsiltLE NO 

To Obtain To Obtain Response 

When 

Needed 



10 :i 0 

None available at lime of evaluation 
8 I 0 

10 0 0 

6 0 0 
None available at lime of evaluation 
110 0 



0 



0 

0 

0 



8 

6 

II 



0 



15 



Indicate 


the overall degree of effectiveness you feel the 


transparencies 


had in achieving 


the lesson 


objectives. 
















VERY 








BELOW 


VERY 


NO 




HIGH 


mml 




AVERAGE 


AVERAGE 


LOW 


RESPONSE 


Unit 1 


2 


3 




6 


0 


0 


5 


Unit //' 


None 


available at 


time 


of evaluation 








Unit III 


2 


6 




1 


0 


0 


8 


Unit IV 


3 


7 




1 


0 


0 


6 


Unit V 


3 


4 




0 


0 


0 


10 


Unit VI 


None available at 


time 


of evaluation 








Unit VII 


1 


1 




0 


0 


0 


15 
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TABLE I Continued 



SECTION II ■ Slide Series and/or Filmstrips 



I. Indicate the physical condition of the slides and/or filmstrips used. 

EXCELLENT GOOD FAIfi 

IT 



Unit I 
Unit II 
Unit III 
Unit IV 
Unit J 
Unit VI 
Unit VII 



13 



8 



l 



l 



0 



None available at time of evaluation 



0 



0 



None available at time of evaluation 



0 



0 



None available at time of evaluation 



POOR NO 

RESPONSE 

0 3 

0 8 

0 1 * 

0 10 



a. 



Indicate your 


accessibility 

ALWAYS . 
Available 


to a remote controlled , school owned, 2x2 slide projector. 

SOMETIMES DIFFICULT IMPOSSIBLE School 

Not To Obtain To Obtain Does NOT 

Available W1Y Mt u 


Unit 1 


1 


0 


1 


2 


8 


Unit II 


4 


1 


1 


0 


3 


Unit II 1 


None available at time < 


of evaluation 






Unit IV 


2 


0 


0 


0 


1 


Unit V 


None available at time 


of evaluation 






Unit VI 


3 


0 


1 


0 


3 


Unit VII 


None available at time 


of evaluation 







NO. 

REST. 

3 

8 

14 

10 
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TABLE ! Continued 



Indicate your accessiblity to a combination 2x2 slide projector - filmstrip projector. 





ALWAYS 

Available 


SOMETIMES 

Not 

Available 


DIFFICULT 
To Obtain 


IMIWSSIBLE 
To Obtain 


School NO 

Does NOT RESP. 
OWN ONE 


Unit 1 


8 


4 


1 


0 


0 l 


Unit II 


8 


1 


0 


0 


0 8 


Unit III 


None available at time of evaluation 






Unit IV 


3 


0 


0 


0 


0 14 


Unit V 


None available at time of evaluation 






Unit VI 


5 


2 


0 


0 


0 10 


Unit VII 


None available at time of evaluation 







Indicate the overall degree of effectiveness you feel the slide series and/or filmstrips had in 
achieving the lesson objectives. 





VERY 






BELOW 


VERY 


NO. 




HIGH 


HIGH 


AVERAGE 


AVERAGE 


LOW 


RESP. 


Unit 1 


2 


3 


7 


l 


0 


7 


Unit II 


1 


6 


2 


0 


0 


8 


Unit III 


None 


available at 


time of evaluation 








Unit IV 


1 


I 


1 


0 


0 


It 


Unit V 


None 


available at 


time of evaluation 








Unit VI 


3 


3 


1 


0 


0 


10 


Unit VII 


None 


available at 


time of evaluation 
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1. One half of the teachers went on field trips where recommended. 

2. The overhead transparencies did an adequate job with a minimal narration by the 
teacher. This could indicate student involvement in discussions. 

3. The instructional media was motivational. 

4. The immediate availability of the media provided sufficient instructional resources 
when they were needed. 

5. The professor-media specialist relationship worked effectively in correlating the media 
with the lessons. 

6. The teachers felt the media was an important part of the curriculum materials. 

7. Overhead transparencies, slide series, and the filmstrip were all effective with no 
indication of one being more so. This, again, indicated effective planning and design 
of the media. 



TABLE 2 

Responses to Questions Pertaining to Instructional Media 



&«? went on field trips to see enterprises. 

Effective use of overhead requires too much 
narration by me. 

Use of the overhead transparencies retains the 
students attention better then straight lecture. 

I have more inslrnctioiutl resources to use in my 
industrial arts classes. 



SD 

7 

II 



0 



6 



3 

4 



0 



H 



u_ 

0 



3 



5 



_£ 

6 



5 



0 



SA 

4 

0 



12 



\R 

I 

I 



The instructional media was effectively correlated 2 0 

with the lessons. 



3 13 



The instructional media are not an important 13 4 / 

part of the materials supplied by the American 
Industry Project. 

Overhead transparencies were more effective 0 7 10 

than the slide series and film series. 



I I 



3 0 



/ 



/ 



% 
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SUMMARY 



A brief summary of the proceeding materials would indicate the gradual and systematic dev- 
elopment of creative and involving instructional media. To become creative and involving, each 
design; each format; each medium and each intent was closely scrutinized, tested and evaluated by 
the Media Specialist, the Curriculum Specialist, and the all important teacher. 

Evaluation caused changes . Revisions were frequent. The end product then is a complete 
set of unique and challenging instructional media to include slide series, overhead transparencies, 
a film strip, a 16mm film and three film catalogs. 

Looking Ahead 

The American Industry Curriculum will grow in knowledges, techniques, approaches, and 
status. With this advancement the necessity of supporting mediative devices will increase. 

It is quite evident that all media formats will become involved. Slides, overheads, filmstrips, 
video tapes, single concept films, audio tapes, and instructional films will be as vital to the Amer- 
ican Industry Curriculum as they are to any educational program. 

What for the future, then? The American Industry Project has, in fact, been an innovator. 
The creative approaches to the media development were centered around the use of the media. 
Design, in terms of commercial art techniques, were more or less standard. 

The future direction, then, should be one of developing more effective and challenging 
methods of student utilization of the media.. The student must become involved. Further, it is 
not intended that the students merely view the media and respond to it. This has already proved 
to be somewhat effective and still has its place in any program. The phrase "student involvement" 
implies much more; at least in this instance. ’ Student involvement in planning, production, and 
evaluation is necessary. 

How then, could this be implemented? A few ideas are listed here. 

1. Would it be possible to have film footage of various subjects available so that the stud- 
ents could make their own films to suit their needs? 

2. A slide file could be made available to students for utilization in student reports and 
presentations. (It is done in industry, why not in American Industry). 

3. Simple kits for making overhead transparency masters and/or transparencies could be 
made available to students for developing supporting materials for their presentations. 

4. Single concept films could be developed by the American Industry classes. 
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5. Why not have the student public relations department ms?<e a PR siiide series or 
film on the student industry and use it in their advertising campaign. 

Here are just five possibilities. The intent is quite evident. If instructional media does 
increase knowledge retention (as it has proved to do so) then student involvement could possibly 
boost retention to higher percentages. 
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Lesson Analysis and Feedback System: 



Description and Analysis of Data 



'* Introduction 



The design of instructional materials requires a myriad of decisions. Often these decisions 
have to be made on the basis of insufficient information. It is likley, therefore, that some of 
the original judgments will be imperfect. 1 In a continuing developmental program this fact is 
not as troublesome or unnerving as it would be *n the instance where there is no provision for 
revision. During each revision phase, prior decisions can be evaluated and modified. 

Since the American Industry Project was funded for a four year period, there was an op- 
portunity to evaluate, revise, and evaluate the revisions. In some instances, this cycle was repeat- 
ed three or four times for a specific set of materials. Specific procedures were established to 
assure that the information gathering process was systematic, reliable, and valid. 

All of the information gathering techniques and analyses presented in this paper are based 
on the work of the curriculum specialist. Richard Gebhart (1968) has done an effective job of 
presenting the methods used to develop the secondary school courses. Thus, only a partial sum- 
mary of these procedures will be included here. In brief, the task of constructing the secondary 
school courses centered on translating the American Industry structure of the knowledges of in- 
dustry into instructional materials that would establish the concepts of industry, interrelationships 
among these concepts, and the ability to solve industrial problems in the students exposed to 
these materials. 

Gebhart's paper describes in some detail the techniques used for the first level American 
Industry course. And, this paper will deal primarily with information generated from the feed- 
back on this course. 

As mentioned above, the basis for the instructional materials was the Project's structure of 
the knowledges of industry. Using this structure as the framework, knowledges and understand- 
ings related to the concepts and interrelationships were defined. A plan was then developed 
to place these in a logical and effective sequence in the learning materials. After the materials 
were developed, two types of information were obtained on their structure and quality. A les- 
son analysis procedure was designed to determine if the lessons and units in the instructor's 
guide were developed as planned. In addition, the participating teachers were requested to 
complete feedback forms on the instructional materials. The next section describes the lesson 
analysis procedures and exhibits a sample of the information produced. The lesson feedback 
system and a summary of the data are presented in later sections of this paper. 



This is not meant to imply that these decisions should not have been made. Actually, t?.oy 
may have been the best at the time they had to be made and absolutely necessary for progress 
in constructing the materials. Rather, I am suggesting that decisions provide the foci for data 
gathering and evaluation in the development process. 
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Lesson Analysis 



The instructor's guide was designed to help the teacher develop a concept, show its role in 
industry, and relate it to the concepts previously discussed. Also, the lessons were written to 
provide for more sophisticated problem solving as the course progressed. Information, activities, 
and problems were selected and placed in the guide to achieve this design. 

Several writers worked on the lessons during the 1967 summer session and part of the 
ensuing school year. On occasion other duties interrupted the continuity of this work. In ad- 
dition, there were eighteen concepts and the attributes of the Project s definition of industry to 
account for in the eighty-four lessons in the guide. Obviously, there was a need for some means 
to check the product against the original plans and specifications. 

An analysis procedure was developed that would provide data on the content and structure 
of each lesson. The analysis form consisted of a matrix with the concepts and attributes appear- 
ing on the horizontal axis and the major portions of the lesson on the vertical axis. These major 
segments on the vertical axis of the lesson analysis form coincide with the lesson format in the 
guide and are in order: introduction, body, summary, and lesson activity. Within each of these 

categories there are sub-divisions to indicate the high and low taxonomy levels and the number of 
instances at each level. In the introduction and body sections, there are spaces to record the 
level of the problems and activities included in the materials. The second section of the form 
provides a means to record the interrelationships made between concepts. The numbers recorded 
in the cells formed in the matrix represent the six levels in the Taxonomy for the Cognitive 
Domain (Bloom, 1956). A sample of the Lesson Analysis Form is reproduced in Appendix F-A. 
Numerical labels for the con"*pts and attributes are listed in Table 2. 

In general, the analysis was completed in the following manner. The reviewer analyzed the 
lesson and recorded the low and high taxonomy level observed in each portion of the lesson 
and the level of the problems and activities. The number of instances at each level was also 
determined. After this information had been recorded, the relationships made between concepts 
were listed in the second part of the analysis form. An analysis form was completed for each 
lesson in the guide. 

After the analysis had been completed on the guide, the information was placed on t?b 
cards. A card was punched for each line in the analysis form. The last seven columns on each 
card contained numbers that identified the level of the course, the unit, the lesson, and the card 
sequence within the lesson. Thus these cards could be sorted and assembled in any manner 
deemed necessary by the researcher and still be identified as to the lesson, unit, and level to 
which they pertained. 

A printout of the cards sequenced by lesson gives an outline of the content of the lessons. 
Table 1 gives an example of the output for the first lesson on communication. Sorting on a 



2 An 'instance” is a unitary bit of instruction that can stand by itself. A specific example, 
problem, or description is an "instance.” 
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Lesson Analysis Output I'nit 
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Halt Concepts 



Numerical Labels for ihe American Industry 
Structure and Definition 



Numerical' Lafud Concept 



1 Communication 

2 Transportation 

4 Finance 

5 Property 

6 Research 

7 Procurement 

8 Relationships 

9 Marketing 

10 Management 

11 Production 

12 Materials 

13 Processes 

14 Energy 

Ring Concepts 

Njunerical Lai* * Concept 



20 Government 

21 Private Property 

22 Resources 

23 Competition 

24 Public Interest 

Attributes of the definition of industry: 
Numerical Label Concept 



40 Institution in our society ( also indicates the entire definition .if 

industry) 

41 Applies knowledge 

42. Utilizes resources ( human and natural) 

43 Produces good and services 

44 Meets needs of man 

45 For monetary profit 
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(filler basic terms used: 



Numerical Label 


Concept 


oO t i • • • • 

.5/ 

60 

70 


Attributes, characteristics, or similarities 
Concept, conceptual 
Productivity 

Student booklets and / or reading materials 
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Sorting on a given concept or attribute produces a listing of all of the lessons in which that item 
appears. For example, the listing for the communications concept identified forty-two lessons in 
which material related to this concept is presented. 

Table 3 gives some excerpts from this printout. Part I contains the non-zero items under 
communication from the output presented in Table 1. That is to say, it contains the information 
on the content related to communication presented in the first lesson of Unit 3. Lesson and 
Unit numbers are contained in the last seven numbers at the extreme right hand side of the print 
out. Reading from right to left, the first three numbers represent the card sequence within les- 
sons. The next two digits indicate the lesson number. In this example, it is 01 or lesson num- 
ber 1. The next number, sixth digit from the right, lists the unit number. In this case, it is 
3 or unit number 3. The last digit in the sequence gives the level of the course. 

Part II presents the information on the fourth lesson in Unit III. A brief review of the 
numbers in the first column to the right of the verbal descriptors discloses that the introduction 
presented factual information related to communication. The body of the lesson started at this 
level also and moved to the application of this material. Application was also involved in the 
activity used in this portion of the lesson. In the summary, factual information was reviewed 
and the student's comprehension was checked. 

Part III is the interrelationships matrix from lesson four in Unit V. This information can 
be discerned in the numbers given in the last seven columns. The large card sequence numbers 
listed in the last three columns place these cards in the interrelationships matrix. The lesson 
and unit numbers given to the left of the sequence numbers further identify the source of these 
data. A look at the numbers given in the first column to the right of the verbal labels reveals 
that communication is related to marketing in this lesson. (Marketing is identifed by the 09 to 
the left of the verbal descriptors.) Moreover, communication is discussed through the synthesis 
level and correlated with marketing. If the reader were to review the actual lesson, he would 
find that the students were asked to apply their concept of communication to the development 

of an advertising program. 



Comparing the data in parts I, II, and III in Table 3 discloses that the lessons on comm- 
unication require higher levels of thinking as they progress in the guide. Note that only the first 
three levels are specified in lessons one and four in Unit III while lesson four in Unit 5 includes 

the synthesis level. 



Another brief look at the remainder of the matrix presented in Part III of Table 3 reveals 
that other concepts and attributes were also related to marketing. Finance (No. 4), Research 
(No. 6). Materials (No. 12), goods and services (No. 43), needs of man (No. 44), and the stud- 
ent booklet (No. 70) were assoicated with the marketing concept. 

Use of this analysis technique has allowed the research specialist to check the structure and 
sequence of the instructional materials developed for the teachers. The flexible manner m which 
the data cards can be sorted provides ready access to the information required in evaluating and 
revising the instructor's guide. A computer program could be written to perform these sorts and 
produce a more readable output; however, since the present output can be rather easily interpret 
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Lesson Analysis Output: Selected Output for the Communication Concept 



ed by its author, the decision was made to place a low priority on the development of this 
program. 

The volume of information generated by this analysis was rather large and of specific 
interest to a small audience. Hence, it was not reproduced in this report. The American Indus- 
try Project, however, does retain copies of the output. 

Analysis of the data generated by the lesson analysis indicated that the objectives specified 
in the original plan for the guide were adequately met. Some omissions were observed and steps 
have been taken to revise or add to the lessons to remedy these problems. Feedback from the 
teachers, which is discussed in the next sections, also confirmed this finding. 

Description of the Feedback System 

The American Industry instructor's guides were designed and constructed to aid the American 
Industry teachers. In fact, their comments and experiences were a valuable input into the plan- 
ning of these guides. For example, the inclusion of more content, activity sheets that are easily 
reproduced, and the integration of the narrative for the media in the guide came as a result of 
suggestions by the participating teachers. Thus, feedback from the practitioner had provided 
valuable data. However, after the last revision of the Level I guide, there was a need for more 
specific information upon which to base the more subtle refinements that would be required in 
the future. In other words, future decisions would require more definitive data than the general 
information produced by the original, somewhat informal , feedback system. 

An analysis of the judgments that would be required in further developmental work on the 
guide assisted in delineating the characteristics of the new feedback system. Information had to 
systematically collected on appropriate sub-sets of the instructional materials. This suggested that 
feedback should be related to each lesson and Unit. Larger segments would be difficult to rate 
and would provide less specific data. 

Nature of the Feedback Forms 



In addition to being system* tic and precise, the system had to provide valid data for the 
decisions encompassed in curriculum development. Hence, the lesson and Unit feedback forms 
were designed to collect information on such factors as the amount of time consumed by the 
lesson, the quality of the content outline, and the value of the activities provided. Also, the 
curriculum specialist was interested in the sequence of content from lesson to lesson and the 
appropriateness of the unit structure. 

The complete lesson feedback form is presented in Figure 1 and the unit feedback form is 
reproduced in Figure 2. Note that there are response statements to check and a place to add 
comments. This format provided specific data that the curriculum specialist needed and allowed 
the person completing the form to add his own judgments and reasons for his responses. An- 
other important characteristic to note is the fixed responses for most of the items do not fall 
in a continuum. Rather, they were written to provide specific information. For example, item 




LEVEL II 

LESSON EEEIHUCK 



( Jirriftthim Feedback Inf nr motion 
American Industry lien. '2/ 20 f OH 



Now that you have romp In led this lesson, please complete this form and nuiil it to Hiehard 
Cchhart . American Industry Project. 

f. heck the appropriate statements and comment as requested. Note: In some cases more 

than one statement may be checked. 

If you need more room for comments , attach additional pages. 



Instructor 




Date 




Unit Number 


Lesson Number 




Date Less Started 



1. Amount of time used to complete lesson (to nearest V 2 class period ) . - , . 

2. Overall , this lesson is ranked as (a)\ 1 excellent , (b)\ I good, (c)\ [ fair, 
(d) \ | poor. If (c) or (d) explain 



3. The objectives of this lesson are (a) n clearly defined, (b) | 1 adequate 

(c) Q not clear, (d) □ too broad, (e) n too brief. If (c), (d), or (e) explain. 



I. The scope in this lesson (a) D described the lesson clearly, (b) □ did not describe 
the lesson. If (b) explain — - - 



. 5 . The content outline (a) f~~l was logical, (b) p contained adequate detail. 

<<>u was not logical, did not contain enough information. If (c) and! or 

(d) explain . 

6. Which activity did you use ? 

The activity used in this lesson (a) 1 t was adequate, (b) 1 1 was not adequate, 

(c)\—\ was developed by the instructor. If (b) or (c) explain - 

(He sure to fill out the items on the reverse side.) 

Fig. I Level I Lesson Feedback Form: 1967-68 Version 

F*9 



O 





LESSON EEEDIIACK 



7. The evaluation items (a) tZ3 M>ere adequate, (b) [2H were not enough of them 

It-} j [ were unrelated to the lesson , (d) p~1 none in the lesson, fe) □ were developed 

by the instructor. If ( c ) explain, if ( e ) send copies of the items 

H. This lesson (a) CD fit Into the sequence of other lessons in this unit, (b) | | did not 

fit into the lesson sequence. If (b) explain _ - _ 

9. The suggested student reading from the student booklets (a) £3 helped to meet the ob- 
jectives of the lesson, (b)\ I had little relevance to the lesson, (c) | ~1 was not used. 

If (b) or (c) explain . ■ » — 

10. The suggested outside references (a) were used and found helpful (identify), 

(b) \ 1 were not used, (c) □ other outside references were used. If (a) or (c) explain 

or list the references. - - — - 

11. The media suggested for this lesson was (a) f I effective, <ba ineffective 

(c) £3 none used. 

12. Did you develop any of your own media Y (a) yes, (b) no. If you did, was it 

effective ? (a) \ | very effective, (b) £2] effective, (c) a ineffective u>n poor. 

Send a copy or description of your media to the Project. 

Id. List any films used with this lesson and rale them as (E) excellent. (G) good, (E) fair, 
or (P) poor - 
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i.f.vfl n 
1 1 mi tmifJlMA 



Curriculum Feedback Information 
1 nif fir n a I ruins try H 2 1 20 f (til 



\ () n) that vi m hai r eotnph ted this unit , pleura* complete this form and mail it to Ha hard 
Gebhurt. American Industry Project. 

Check the appropriate statements and comment as requested. Note; In some cases more than 
one statement may be checked. 

If you need more room for comments , attach additional pages. 

Instructor — Date , . , . — — 

Unit Number Date Started Date Completed 

1. The approximate time spent in laboratory ivas %. 

2. Generally , this unit was found to be (a)\ ] excellent , (b) 1 ~1 good , (c) [__| fair, 

(d) n poor. If fc) or (d) explain. 

3. The general unit objectives were clearly defined , (b) tZD adequate (c ) tZJ not clear, 

(d) 1 1] impossible to meet , fe) tZD too broad, (f) L-J too restrictive. If fc), (d), or fe) 

explain — — ■ < — — ■ 

t. Generally, the activities were found (a) f | adequate, ( b) □ not suited to the lessonfs), 

fc) not enough of them. If fb) or fc) explain ■ ■ • - - — 1 — — * — 

. 5 . The student booklet fa)\ 1 provided enough background reading for the students, fb)£3 
should have contained more information WD did not fit together with the lessons. 

If fb) or fc) explain . . ■ ■ ■ 

6. The instructional media for this unit was generally fa) £3 effective, fb) Q ineffective. 

fBe sure t'' f'U out the items on the reverse side.) 



l''ig. 2 Level I Unit Feedback Form: 1 W> 7 -Ml Persian 
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I IV IT FEEDBACK 



Give a general summary of your feelings about this unit. 



Include the strong and iveak 



points you found. 
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five on the lesson feedback form has the following response categories: a. "was logical", b. 

"contained adequate information". c"was not logical." and d. "did not contain enough informa- 
tion." Thus, if there were problems with sequence and detail in a lesson, the teacher could rate 
this. Moreover, these response choices focus the responder's attention on these facets of the con- 
tent outline. In contrast, an ordered series of response positions, say from SA to SD, would 
not give as specific information on the material rated. If the reader is interested in the format 
and contents of the feedback sheets, these forms can be reviewed in Figure 1 and 2. Summar- 
ies of the feedback data will be presented in later sections of this paper. 

Color coding was used on the feedback sheets to distinguish them from other materials. 

Blue paper was utilized for the first level form. Black and red inks were employed for the les- 
son and unit feedback forms respectively To further facilitate the feedback process, a lesson 
feedback form was placed at the back of each lesson and a unit feedback form was entered at 
the end of each unit when the instructors guides were collated. Hence, as each lesson or unit 
was completed, the teacher would encounter a feedback form. 

Feedback Procedure 



The Project's participating teachers were requested to fill out a feedback form as soon as 
feasible after a lesson or ^nit had been completed. Self-addressed enveloped were provided for 
returning the completed feedback forms to the curriculum specialist at the middle and end of 
each month. After a brief review of these forms, the curriculum specialist sent them to the 
project's key punch operator. 

Two sets of data were punched for each feedback form. Figure 3 presents a card image of 
one record in the lesson feedback system file. The six fields on the left, columns one through 
sixteen, are used to identify the source, level, and type of data recorded in the remainder of 
the card. Unit feedback records utilize a similar format. (See Figure 4). 

After the feedback cards had been punched, they were processed by means of the Total 
Feedback Summary Program. This program summarized and stored the information contained in 
the first six fields. A sample of the output from this program is given in Figure 5. From this 
printout one can identify the data forms and feedback sheets returned by each teacher. When 
these summaries are printed periodically, every two weeks on the Project, the progress and status 
of each teacher can be ascertained. The format of the output can be modified by use of para- 
meter cards to include a maximum of fifteen units and up to thirty lessors in a given unit. 

The information contained in this feedback summary form has been very valuable in mon- 
itoring the progress of the teachers and determining the status of the courses at a given time. It 
helped to identify teachers who were having problems or were slow in returning their feedback in 
time to take corrective action. In addition, it condensed and coordinated a variety of data re- 
lated to the evaluation. As a result, the research and curriculum specialists could quickly obtain 
the information they needed. 

The feedback system also provided a vehicle for the exchange of other ideas. Frequently 
the teacher would attach an activity sheet or test developed for his American Industry students. 
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PARTICIPATING TEACHER'S FEEDBACK SUMMARY 
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36 WKS 



LEVEL I 



STUDENT INFORMATION FORM - N CLASS ROSTER - N 
GRADE SHEET - N OCCUPATIONAL PREF FORM - ft CLASS OPIN ION N AIRE -ft 

END-OF-YEAR REPORT - ft INSTRUCTORS GUIDE -N 

UNIT /-ft UNIT TEST - N NELSON RATING SCALE - ft MEDIA FEE Dll AC K 

MEDIA PACKET -N STUDENT BOOKLETS - N 

LESSON 1 - ft LESSON 2 - ft LESSON 3 - ft LESSON 4- ft 

UNIT 2 -ft UNIT TEST -N NELSON RATING SCALE -ft MEDIA FEEDBACK -\ 
MEDIA PACKET - N STUDENT BOOKLETS - N 

LESSON l-N LESSON 2-ft LESSON 3-R LESSON 4-R LESSON 5-R 

UNIT 3-R UNIT TEST -N NELSON RATING SCALE -ft MEDIA FEEDBACK \ 
MEDIA PACKET -N STUDENT BOOKLETS -N 
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2-R 


LESSON 
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21-R 
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Fig. 5 Feedback Summary Output 
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The flow of this type of information increased with the adoption of the feedback system. 



Up to this point, the discussion of the feedback procedure has centered on the collection 
of data and v the methods used to inventory and store this information. The primary purpose of 
the system, however, was to obtain ratings on items placed on die feedback sheets. These ratings 
and comments are punched in the data fields to the right of the information retrieval fields. 

(These fields are identified in Figures 3 and 4.) After the cards were processed in the Total 
Feedback Summary Program, the ratings were summarized by lesson, unit and teacher. From 
these summaries, the characteristics of each lesson and unit can be determined and the teachers 
response tendencies can be estimated. A graphic summary of the feedback information flow is 
given in Figure 6. The analysis of these summaries will be discussed in greater detail in a mom- 
ent; hoi ever, at -this point consideration of the reliability and validity of the data collected in the 
feedback forms is appropriate. 

Reliability and Validity of the Feedback Forms 

Interpretations of daf are contingent on their accuracy and the degree to which they relate 
to the variable being measured. The first attribute mentioned is concerned with reliability. The 
second deals with validity. Since the rating forms were comprised of a hetrogeneous set of items, 
internal consistency measures of reliability were not appropriate. However, the teachers with 
semester courses completed two sets of feedback forms for the lessons they taught. 

These two sets of data provide the basis for a test-retest comparison to determine reliability. 
More specifically, this comparison is concerned with the stability of the ratings over time. In 
this instance the time period was about four months. Also, it is assumed that the teachers' 
perspectives did not undergo a significant change. For example, after completing the first semester 
the teacher might attain a Gestalt view of the course which could in turn affect his perception 
of specific lessons or units. Several factors mitigated against this event in feedback system. First 
the teachers had been thoroughly exposed to the philosophy and the objectives of the course 
during summer institutes. Second, the course was based on their prior comments and experiences. 
Hence, they should have had the overall view before teaching the first semester course. 

The correspondence between the fir«t and second ratings on the lesson feedback forms can 
be observed in Table 4. Only those items on which a stable rating could be expected are in- 
cluded in the table. Thus, for example, item six was not included since the activities could 
vary from one semester to the next. Items and response categories are given in Figure 1. 

Response "a" is represented by a I in all instances. 

The agreement between the two sets of ratings on all items is sufficiently high to conclude 
that the feedback form produces relatively stable data. Three items of special interest are the 
ones dealing with the general rating of the lesson, item two; the rating of the objectives, item 
three; and the rating of the content outline, item five. On item two, 63 per cent of the ratings 
remained the same and less than 4.1 per cent changed as much as two response categories. Al- 
most 91 per cent of the responses concerned with the objectives for the lessons are the same on 
both the initial and second ratings. Eighty-five per cent of the content outline ratings remained 
the same. 
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Fig. 6. Flow Diagram of Feedback Information How 
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First 



TAliUC 4 

Comparison of First Second Ratings of hcjsons 
Rating PM li(Uin K 



Response 

1 

2 

3 

4 



/ i» 

2* 5 

4 29* 

0 2 
0 0 



3 t 

2 0 

5 0 

0* 0 

0 0* 



ITEM 2 

63% of ratings are 

identical 



First Rating 



First Rating 
Response I 2 



l 



51 



Response I 2 3 4 




90.9% of ratings are 

identical 

First Rating 

First Rating 
Response l 2 3 4 



1 



44 



0 



2 0 



ITEM 4 



92.7% of ratings 
are identical 



2 0 10 0 
3 0 0 1 0 



4 



0 0 0 0 

I TEM 5 

85% of ratings are. 

identical 



First Rating 



First Rating 



Respons eq l 2 
I 55 0 



Response l 



1 



31 



9 



4 



2 



0 0 



2 



I I 



ITEM 8 
100% of ratings 
are identical 



ITEM 9 
86.5% tf ratings 
are identical 



* Numbers in this diagonal represent identical ratings. 



On the basis of the data collected, the responses to the feedback items appear to be stable. 
Certainly their reliability was adequate for the types of decisions that were based upon them. 
Another question, however, needs to be answered before these responses can be used. This ques- 
tion pertains to the validity of the data. 



The feedback system was conceived and implemented to generate data related to the qual- 
ity and operating characteristics of the instructional materials. Therefore, if the ratings are valid, 
they should vary with the quality and characterises of these materials. Three evaluations of this 
relationship were possible in the present set of data. 



First, item seven in the lesson feedback form has provision for rating evaluation items if 
they appeared and for noting that there were no items in the lesson. (A sample feedback form 
is given in Figure 1.) Several lessons in the guide did not contain evaluation items. Hence, for 
these lessons, the "no items" choice should be checked. A comparison of the responses to item 
seven for the lessons without evaluation items and the responses for a random sample of the 
lessons with evaluation items gave the results depicted in the contingency table below. If the 
responder checked any of the first three responses to item seven, he was included in the column 
labeled "Quality of Evaluation Item Rated" since these choices referred to the quality of the 
evaluation items. If he checked response "d" he was placed in the column labeled "none in 
the lesson." The status of the evaluation items in the lessons is represented in the rows. Per- 
fect agreement between the ratings and the status would require that all of the responses would 
fall in the upper left and lower right cells. 



M 



RATING 

Quality of Evaluation "None in the lesson" 
Items Related (Choice d) 



a 

a 

(A 



Has Evaluation Items 

Does Not Have Evaluation Items 



105 

5 



6 

23 



Although the data do not ”eveal perfect correspondence, the (phi) coefficient for this set 
of data is .76. Using the formula, = N2, a value of 80.6 is found for Mays, 1963). 

The probability of obtaining an OO al ” e this large if the ratings and the status of the lesson 
were not related is less than one in one thousand (P-^.OOI). Hence, there is a significant 
statistical relationship between the ratings and the status of the lesson. 



Another check on the validity of the feedback responses involves the comparison of the 
ratings for the lessons in Unit II against those for the lessons in the other units. Unit II 
lessons, based on the history of industry, provided a number of developmental problems. And, 
when they were done it was the opinion of the Project staff members that these were the weak- 
est lessons. Thus, if the information collected on the feedback form is valid, the ratings for 
these lessons should be lower than those for the lessons in the other units. A summary of the 
responses for item two in the lesson foedback form is given below. Column "a" represents a 
rating of"excellent" v and "d" is "poor." The percent selecting each rating is given in the paren- 
thesis. 
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Response Categories 



Unit abed 

II 6(14.6) 25(61 ) 8(19.5) 2(4.9) 

I & lll-VII 168(32.3) 311(59.8) 33( 6.3) 8(1.5) 

Comparing these distributions with the Kolmogorov-Smirnov Two-Sample Test (Siegel, 1956) 
disclosed that the probability that the ratings for Unit II were from the same distribution as the 
ratings for the remaining Units is between .10 and .05 ( .10 ^P ^.05). Therefore, the ratings 
appear to be somewhat sensitive to a variation in quality. 

The average unit ratings for each of the seven units in the first level course are graphically 
depicted in Figure 7. These averages are based on the teachers' responses to item two in the 
unit feedback form (Figure 2) for each unit A comparison of the responses, number and per 
cent, for Unit II and the remaining units is presented below. 



Response Categories 



Unit 


a 


b 


c 


d 


II 


0(0) 


9(69) 


3(23) 


1(8) 


1 & lll-VII 


16(39) 


21(51) 


4(10) 


0(0) 



Using the Kolmogorov-Smirnov Two-Sample Test, it was found that the probability that 
these two sets of ratings came from the same distribution is less than five chances in one hund- 
red (P ^.05). 

Both of the validity tests on the lesson feedback indicate that these forms return valid 
data. The validity test for the unit feedback form also was positive. Unfortunately, there 
were not enough unit feedback sheets from the semester programs to determine the reliability 
of this form. However, since the unit form is very similar to the lesson feedback form, there 
is no reason to believe that it is not reliable. 
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UNIT 



Fig. 7. Average Unit Ratings for Level I 
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A Definition of the Ratings 



Before the data from the feedback forms are analyzed and discussed, it is important that 
the meaning of the ratings be defined. Responses on the feedback forms are conceptualized in 
the following formula. 

R - f(A.Q. + R.R.T.) 

In this formula, R represents the rating or response, A.Q. is the actual quality of the material 
rated, and R.R.T. is the responder's rating tendencies. Since R is the rating, it will be found on 
the completed feedback forms. R.R.T. is a measure of the responder's tendency to be an easy, 
average, or difficult rater. For example, a lesson might be rated as "good" by a lenient rater 
and "fair" by a conservative rater. Yet, the quality of the lesson remains the same and it would 
most logically be located between these two ratings. 

Assuming that later response tendencies are random and normally distributed around X.. 
for our sample of participating teachers, R.R.T. for a given rater can be estimated by X 
rater i. X?. = ,n other words, the estimate of Rj. R.T. is the difference between 

the average rating given all lessons by a rater, X rater j , and the average rating given all lessons 
by all raters, X.. . In X .. the sum of the rater response tentencies is zero since the easy 

and concervative ratings cancel each other. 

A second assumption encompassed in Rj.R.T. * X Rater { -X.. is that the interaction of 

lesson quality and teacher background in normally distributed around Xp ater j. In other words 
some lessons may be difficult because the teacher lacks the proper background while others are 
very easy for the inverse reason. The fact that the first :eval American Industry course deals 
with gross concepts and skills lends credence to this assump:,an. Moreover, the teachers had 
in-service training which should also reduce the interaction factor. 



3 

This assumes that there is no interaction between the teacher's knowledge end abilities and 
the nature of the lesson. In practice this probably will not be true. However, A.Q. as defined 
in the first formula given above will include this interaction component. Calculation of R.R.T. 
will show why this is true. 
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The actual quality of lesson indicated by a specific rating is: 

AQ i1 * R n - Rj-RT. 

Assuming that the *^Rj.R.T. = 0, then the actual quality of a lesson 1 i^^R^ or the average 
of the ratings for the lesson. N 

The preceding formulas apply primarily to items measured on interval or ratio scales and 
to a lesser extent to ordinal scales. Item two on the lesson feedback form and items two and 
seven on the unit feedback form involve measurement at the ordinal level or higher. However, 
the rater response tendency, R.R.T., found on these items will give a general indication of the 
response tenoencies on the other items in the forms. 

For those items in which the response categories do not lie on a continuum, the response 
frequencies for each response position found in the total summary of the ratings for all lessons 
or units can be utilized as the criterion. Item five in the lesson feedback form provides an 
example of an item in which the responds do not constitute a continuum. The total frequencies 
and associated percentages (in parentheses) for all of the responses to this item are given below. 

Response 

abed a+b a+d c+d 

442(80.2) 48(8.7) 11(2.0) 35(6.3) 13(2.4) 1 (.2) 1 (.2) 

The relative quality of the content outline of a given lesson can be assessed by comparing 
its rating to the response distribution above. A large number of low ratings indicate a need to 
review the lesson. However, even o .e low rating my be the signal to review the lesson if that 
response was given by a very perceptive teacher. Usually though, one or two low ratings indicate 
that the raters may require some assistance with the material involved. 

Up to this point, the lesson ratings have been analyzed on the basis of rater response 
tendencies, R.R.T., and the actual quality, A.Q., of the lesson or unit as perceived by the rater. 
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These ratings represent the teacher's opinions of the lessons and units. Or, to look at this in 
another way, the variable being measured in teacher opinion. This variable is most likely com- 
prised of factors that relate to types of instruction typically used by the teacher, his ability to 
adapt to structured materials and courses, his educational preparation, his work experience, his 
interest in innovation, and the way his students react to the material. For the most part, these 
ratings give little evidence on the quality of the concepts and rationale that undergird the course. 
This information will come from a logical analysis of these elements and measures of the out- 
comes that result from the study of American Industy. 

Results and Discussion 

As evidenced by the data and forms presented in the previous discussion, a large variety 
and volume of information were collected through the feedback system. Since detailed analyses 
of these data are primarily of value to the curriculum specialist, the ensuing comments will be 
concerned with reports on the more general characteristics of the instructor's guide for the first 
level American Industry course. Selected uses of these data in future revisions of the material 
evaluated and the developmental process will also be discussed. 

A.Q. of Lessons and Units 

The actual quality (A.Q.) of the units and lessons are listi i in Table 5. Values in the 
table are based on responses to item two on ihe lesson and unit feedback forms. On both 
forms, this item requests the rater to indicate the general or overall quality of the material being 
evaluated. The response categories on this item "excellent", "good," "fsir, ' and "poor" were 
represented by numbers one through four respectively in the calculations of the A.Q. s.3 Hence , 
a lesson or unit of high quality will have an A.Q. with a low numerical value with the maximum 
rating being one. 

Table 5 also lists the number <N) of ratings summarized to obtain each A.Q. In general, 
as N increases the A.Q. associated with it should become more stable. Several factors influenced 
the N's found in the table. Semester programs used only a portion of the lessons in Units II 

N 

J A.Q. for lesson 1 

i il 
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through V!. Some teachers did not have time to complete Unit VII. And. some teachers failed 
to return the feedback sheets. This failure was especially hard to correct in Unit VII since it 
came at the close of the school year. As a result, the reader will have to consider the size of 
the N in each A.Q. as he reviews the ratings. 

The data in Table 5 were reviewed and analyzed to provide the information required to 

answer the two basic questions emcompassed in developing and revising the curriculum materials. 

1. What segment of the instructional materials is most in the need of revision? 

2. Which combination of curriculum development efforts will have the maximum influence 

on the total instructional system? 

The first question can be answered by isolating the largest A.Q.'s in the table. (Note 
the A.Q.'s for the lessons one through four in Unit II and the Unit II A.Q. for example.) In 
addition, a sequence of high values may indicate a topic that is being treated less effectively than 
the other topics in the guide. Other revision candidates include the lessons and units with A.Q. s 
above the average rating for all lessons. 1.79. and all units. 1.83. For example, an A.Q. of 2.00. 
although it indicates a rating of "good" on the scale, is based on the response ; of several teachers 
and these may contain a number of "fair" and "poor" ratings. In practice, then, high A.Q. 
values provide clues to thu lessons or units that need further review. An analysis of the response 
frequencies in each response category for the materials in question will reveal the number of low 
ratings given and a study of the comments on the feedback forms for the lesson or units will 
further define the problem if one exists. 

Another way to conceptualize the A.Q. values is to view them as sample means from a stat- 
istical quality control procedure. Assuming that the average quality of the lessons is X, any A.Q. 
that exceeds X + 3 '* significantly lower in quality than the average quality of the lessons 

(Burr. 1953). To illustrate this, consider the A.Q. of 2.00 for lesson two in Unit II. The control 
limit for this sample is 1.79 + 3J|4) * 2.43. Since the A.Q. is less than this value, the les- 
son would be within the control limits. However, the series of four above average A.Q. values 
for the first four lessons would signal possible problems in these lessons. 
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■ Analysis of Responses to the Remaining Items on the Forms 



The ratings and comments listed for items three through eleven in the lesson feedback form 
and three through seven in the unit feedback form provide information on specific aspects of the 
materials being evaluated. (Copies of these forms are given in Figures 1 and 2). Analysis of the 
responses to these items can help to isolate specific causes of low ratings given to the general 
quality of the materials (item two in the forms). Also, it is conceivable that the general quality 
of a unit or lesson may receive a favorable rating while one or more of the specific qualities may 
be a guide to correcting the materials with the most serious defects; however, more subtle refine- 
ments will require an item by item analysis of the responses to the specific items in the feedback 
forms. 

Since the response categories for item three and those after it do not constitute a continuum, 
the A.Q. procedure used for item two is not appropriate. Instead, the response patterns to each 

item will be utilized and the summary of responses for all lessons and units will be used as the 

criterion. Consider, for example, the comparison of the ratings of the objectives in lesson two 
of Unit II to the total response pattern on this item. Item three in Table 6 presents the summary 
of all of the responses to this item. Whereas 3.4 per cent of the total responses indicate that the 
objectives are too broad or not clear, twenty-two per cent of the responses on the feedback for 
lesson two in Unit II list this rating. Similarly, on the item concerned with the content outline, 
item five, 6.7 per cent of the total responses indicate that the content does not contain enough 
detail and 11.1 per cent of the .atings on lesson two indicate this defect. Thus, these specific 
ratings confirm the problem indicated by the A.Q. for lesson two and suggest two possible causes 
for the low rating. 

Statistical quality control procedures can also be applied to these items. Since they are 
concerned with the attributes of lessons and units, the fraction defective procedure can be employ- 
ed to analyze the response data (Burr, 1953). The data in Tables 6 and 7 can be used to cal- 
culate the proportion of ratings in each "defective" category listed for items three through eleven 

and three through seven on the lesson and unit rating forms. A "defective" category is one that 

indicates a limitation of the material rated, for example, response "d" in item three on the lesson 
feedback form indicates that the objectives were too broad. 
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Summary of Resp<tnsts on the / .esson b eedhack bonus 
For the Level I Instructor's (initio 



'statement Rated 







o 


a 


•7 


Overatl * this lesson 
is ranked 


34 


174 


3. 


The objectives of this 
lesson are 


33 


526 


/. 


The scope in this 
lesson 


35 


542 


5. 


The content outline 


44 


442 


6 . 


The activity used in 
this lesson 


204 


284 




The evaluation items 


92 


274 


8 . 


This lesson 


49 


531: 


9 . 


The suggested studen * 75 
reading from the student 
booklets 


3 77 


/ 0 . 


The suggested outside 
references 


99 


36 


11. 


The media suggested 
for this lesson . . . 


79 


263 


12. 


Did you develop any 
of your own media? 


85 


64 


13. 


List any films used 
with this lesson ■ . . 


541 


48 



O 




Rating 

b v d «* / A' 

336 41 10 

16 16 4 

17 



48 


II 


35 


13 


1 


35 


50 


13 






70 


*> 


II 7 


32 


3 



R 

53 86 

448 6 3 I 

12 241 

443 

5 1 
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Table 7 



Summary of Responses on the Unit Feedback Forms 
For the Level I Instructor's Guide 



Statement Rated 



0 



Rating 



2. Generally , this anil was found to be .... 

■1. The general unit objectives were / 

/. Generally, the activities were found 2 



•>. The student booklet 3 

6. The instructional media for this unit was 

generally 9 

7. General comments on the unit (tone) ... 6 



16 30 7 

50 I I 

39 1 7 

35 13 7 

43 13 2 

7 26 ft 6 
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Determination of the control limit for the rating of the content outline for lesson two is 
done as follows. The fraction defective, p. for the content outline lacking enough information, 
choice c, is the number of responses in category c divided by the total number of ratings given 



to this item, _^c . In this example, p = .067. The control limit is given by p + I 



For lesson two, the control limit is .067 + 3 .0671.933) which equals .092 or 9.2 per cent. 

9 

Since 11.2 per cent of the ratings for this item on lesson two fell in this "defective" category, 
the amount of information in this outline is not adequate. 

The previous paragraphs have demonstrated a general and statistical approach to the analysis 
of the information collected on the feedback forms. Choice of an analysis method will depend 
on the amount of sophistication required. If there is limited time and resources available to re- 
vise materials, the general approach will allow one to determine the segments of the guide that 
are most in need of further input. The general approach has been used on occasion by the 
research specialist and has produced satisfactory results. 

Decision-Making Based on the Feedback Data 

An important i actor to remember when reviewing the tables and figures in this paper is 
that data do not make decisions. People make decisions and, hopefully, these decisions are based 
on appropriate information. Therefore, poor ratings for a given lesson or unit do not necessarily 
mean that revision of thi- material is the appropriate course of action. Other evidence may in- 
dicate that the material u> logical and correct. Hence, it may be the teacher that needs revision. 
An illustration of this is found in the rating of the content outline for lesson two in Unit II 
which deals with the history of industry. The outline could be made very detailed or the teachers 
could receive some in-service training on this topic. The first course of action will solve the prob* 
lem with the lesson in question. The second approach will make the teacher more competent in 
the general area and, thus, should facilitate the utilization of the other lessons on history. 

Before leaving the decision-making discussion, it should be noted that positive ratings do not 
constitute absolute evidence that a lesson or unit is the best means to reach a stated objective. 
Other inputs are required. Student performance, content validity, and logical consistency are 
equally important. 

A review of the data in Tables 5, 6 and 7 will give a general indication of the quality of 
the instructional materials in the first level instructor's guide. The A.Q. values in Table 5 indi- 
cate that Unit II, lesson seven in Unit III, and lesson twenty-three in Unit IV require further 
investigation and possible revision. Considering the number of lessons in the guide, the small 
number of negative ratings is impressive. In addition, the average ratings for all lessons, 1.79, 
and all units, 1.83, support the quality of the instructor's guide. 
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Analysis of Lesson Ratings 



Analysis of ratings for the remaining items in the lesson feedback forms confirm this pos- 
itive evaluation of the lessons. Less than 4 per cent of the responses indicate that the objectives 
for the lessons are not clear or are too broad. When the responses for the lessons are not clear 
or are too broad. When the responses for the lessons in Unit li, lesson seven in Unit III, and 
lesson twenty-three in Unit IV are subtracted from this total, this figure falls below 3 per cent. 
And, only two of the remaining lessons in the Level I Instructor's Guide receive as many as two 
responses in this category. Obvisouly, there are few problems associated with the manner in 
which the objectives are stated in the guide. 

A review of the responses on item four in tha lesson feedback revealed that the lesson 
scopes were concisely written. Approximately 97 per cent of the responses agreed that the scope 
clearly defined the lesson. Only lesson two in Unit II received as many as two negative ratings 
on this item. In addition, the teachers' comments disclosed that the scopes were a valuable part 
of the lessons. 

A study of the data for item five indicated that about 8.7 per cent of the responses rated 
the content outline as not having enough information or not being logical. Omitting the lessons 
in Unit II, so single lesson received more than two negative ratings. Also when the ratings for 
these lessons are removed from the totals, the number of negative responses to the remaining 
lessons drops below 7.5 per cent. The scattered pattern of the negative responses on this item 
when Unit II is no f considered suggests that supplementary information or instruction for individ- 
ual teachers would be a more efficient approach than revision of the lessons. 

Comments and ratings made by the teachers indicated the need for more activities to bring 

out the concepts. In fact, a number of the teachers developed supplementary activities, response 

c, to facilitate their instruction. Response to the activities included in the lessons, however, was 
quite favorable. Almost 75 per cent of the responses rated the activities as being adequate. 

Most of the teachers were satisfied with the evaluation items. About 80 per cent of the 
responses were favorable. Less than 1 per cent of completed forms include a response to the 
effect that the items were not related to the lesion. Several teachers commented that more 
items should be included and that unit tests were very useful. This obviously is an area in which 
substantial improvement can be made in the guide. 

Eight responses, about 1.5 per cent, denoted that a lesson was out of sequence. Four of 

these dealt with the sequence in Unit I. On the basis of these data and comments from the 

teachers, the sequence was revised to parallel the use of the student enterprise activity in Unit I 
with lessons two and three. Subsequent feedback disclosed that this modification worked effect- 
ively. 

Slightly more than 12 per cent of the responses related to the use of the student booklets 
indicated that they had little relevance to the lesson (Item nine). Frequently this rating was 
accompanied by comments suggesting that the lessons should be based on the booklets. In ad- 
dition, one-sixth of the "not relevant" ratings were contributed by two teachers who had classes 
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comprised of disadvantaged students. The first cause of negative responses requires instruction in 
the function of the booklets. The second gives evidence of a need for alternate reading mat- 
erials for special groups. When lessons contained specific page reference* in the student booklets, 
these reading assignments were used in approximately 97 per cent nf the cases. And, frequently 
when specific pages were not identified the teachers complained chat the ,s were not corre- 
lated with the booklets. The value of specific reading assignments is apparent. 



Many lessons contained references to books, periodicals, and pamphlets in addition to or in 
place of assignments in the student booklets. Examination of the responses to item ten reveals 
that about 7.9 per cent of these references were used by the teachers. Responses to a random 
sample of lessons were analyzed further to determine the frequency of use of various types of 
references. In general, the outside references can be classified in two categories based on their 
availability to the classroom teacher. Some of the items, such as newspapers and popular mag- 
azines, were readily available. Others, such as technical books and journals, would require special 
ordering or use of the school library. The contingency table presented below shows the relation- 
ship between usage and availability. 



Rating 




Not readily available 
Readily available 



Were used 
8 

6 



Were not used 
90 

34 



A chi square test indicated the possibility of a al'ght degree of relationship between avail- 
ability and use, .25VP^.10, but this value did not cttain the levels of significance commonly 
used to judge a relationship as being significant. In addition to the low utilization of the refer- 
ences listed, slightly less than 2 per cent of the response forms indicated that the teacher had 
found and used outside references other than those listed. The low rate of utilization of refer- 
ence materials from common and technical publications presents an obvious problem for these are 
some of the best sources of current information and thinking on industry. In addition, thev pro- 
vide real-life referents for the ideas discussed in class. Additional work is needed to encourage 
the teachers to make use of these resources. 



Approximately 4.4 per cent of the ratings on the media provided in the instructional mater- 
ials package for the first level course evaluated the instructional media as being ineffective (Item 
eleven). The * dia for lesson three in Unit I were the only ones to be related low by more 
than one teacher. Most of these negative comments pertain to one of the original slide series 
done by the Project. From this rather gross assessment, the media appear to be a valuable part 
of the instructional materials provided to the teacher. John Zuerlein's (1969) paper discusses the 
nature of the media and the teachers' evaluations in greater depth; thus, this paper will not go 
into detail in this area. 

The last two items on tne lesson feedback form pertain to the number of media developed 
by the teacher and the number of films used in conjunction with the lessons. About 13 per 
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cent of the response forms noted that the teacher had designed and produced his own media. 

Most often these were in the form of overhead transparencies. A variety of films were projected. 
The primary problem with these was to schedule them at the appropriate time. Because of this 
scheduling problem, little emphasis was placed on the use of films in Units III through V. Since 
the content of the remaining units is more general, the scheduling problem was diminished for them. 

Analysis of Unit Ratings 

In prior pages, the overall rating of the units has been discussed and analyzed. Thus, it is 
not necessary to restate that material here. Before going on to item three, brief mention should 
be made of the responses to the first item on the feedback form. This item asked the responder 

to list the amount of time spent in the laboratory (See Figure 2). The lack of systematic feed- 
back of the unit forms makes the data on this item very difficult to interpret. In general, the 
responses to this item and comments by the teacher indicate that there is a need to develop 
more effective use of the laboratory --especially as it relates to the development of concepts. 

Several teachers, for example, complained of the problem they had as their students went into 
production in the shop and suggested that a unit on shop orientation should open the course. 

One should expect problems starting a new production run. In fact, industry typically encounters 
these problems. That is the reason for the pilot production run. What better opportunity exists 
for teaching the importance and general techniques involved in planning, following plans, working 
together, and operating equipment? 

Interpretation of the response patterns on the remaining items on the Unit feedback forms 
is also hampered to some degree by the lack of responders. However, when the responses are 
pooled for all units some observations can be made about the general unit structure in the course. 
Data upon which the ensuing comments were based are reproduced in Table 7. 

A high rating was given to the unit objectives. The sole negative response was given to 
Unit II. With the exception of Unit II, there appears to be little room for improvement here. 

Most of the responses, 78 per cent, indicated that the activities included in the units were 
adequate (Item four). Several teachers also noted the need for more activities. This coincides 
with data and comments made in previous parts of this papet. There is an obvious need to 
design lab activities that develop the concepts of industry. 

Seventy per cent of the responses listed the student booklets as providing adequate back- 
ground reading for the students. Twenty-six per cent suggested that they should have contained 
more information. Only 4 per cent indicated that the booklets did not fit together with the 
lessons. Interpretation of these responses depends on the assigned function of the booklets in 
the instructional process. They were written primarily to portray an industry in action and pro- 
vide referents for the ideas discussed in the lesson. However, some teachers thought that they 
should be "how-to-do-it" b jks. There may be a need for a separate manual to present the 
"how-to." 
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Assessment of the media in the units was very favorable. This substantiates the media 
ratings given in the lesson feedback form. Apparently, the decision to emphasize overhead tran- 
sparencies was favorably received. 

Item seven in Table 7 gives the- rating of the tone of the comments submitted by the tea- 
chers. Category "a" contains the numfc.r of forms that listed highly favorable comments. Cat- 
egories "b", "c’\ and "d" represent favorable, neutral, and unfavorable respectively. Four of the 
six negative ratings were given to Unit II. Thus, the remaining units received a favorable rating. 

Special Analyses 

During the design of the instructor's guide for Level I, special provisions were made in 
order to secure information related to three questions of interest. First, the curriculum specialist 
was concerned about the amount of detail required in the content oudine in the lessons. Second, 
there was a need to know the comparative rating of inductive and deductive lessons. Third, 
the research specialist was interested in the orientation of the teachers to the lab work which pri- 
marily involved the making of items versus the lessons that stressed cognitive learning. 

Each of the three analyses to be discussed is based on a subset of the total set of data 
collected. Hence, the statistics presented are probably less stable than those given in previous 
portions of this paper. Thus, the results should be interpreted as "insights" rather than proven 
facts. The logical course of action following from these "insights" would be to replicate them 
with new materials or experimental settings. 

In each of the analyses, lessons containing the two attributes of the variable in question 
were identified and the ratings for these lessons were compared. To control for the differences 
in teacher knowledge and experience associated with the various concepts, lessons exhibiting the 
two attributes of interest were selected within the same concept areas. In the comparison that 
follows, for example, lesson twenty-cne in Unit II had * brief content outline. Since this lesson 
was based on the energy concept, tho detailed lessons employed in the comparison were also 
from the energy area. A similar procedure was utilized in selecting the comparison lessons for 
the other outlined lessons. Lessons used in these analyses came from fo* r different units; thus, 
the teachers' time or background in the course should be accounted for. 

Outlined vs. Detailed Lessons. Development of detailed content presentations is a 
lengthy task and requires careful consideration of the characteristics of the students and teachers 
for which the materials are designed. Thus, it is important to assess the teachers' opinions and 
the students' terminal performance to determine the effectiveness of the two types of lessons. 

This study pertains only to the teachers' opinions. 

When the general ratings of outlined and detailed lessons are summarized the following con- 
tingency table is obtained. The ratings are from item two in the lesson feedback form. 
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Rating ( Item Two ) 



ir 


1 




a & b 


d 


3 


H 


Detailed 


75 


2 


< 

* 


Outlined 


28 


5 



Ratings a and b were combined since they indicate that the lessons were satisfactory. Choice d 
signifies a lack of information in the content outline. 

Although the percentage of responses in the d category for the outlined lessons was relative- 
ly small, fifteen per cent, the Chi-square value for this table is significant beyond the .05 level 
(.05 >P >.02). This value indicates that the outlined lessons received significantly more d resp- 
onses than the detailed lessons. A comment by one of the teachers after his rating corroborates 
this finding as he wrote: "This is another of those skimpy lessons. It hardly deserves rating. 
Based on these findings, the detailed lessons appear to meet with more favor from the teachers. 

Deductive vs. Inductive Lessons. Lessons in the Level I guide were developed in in- 
ductive and deductive styles. There was some concern that the inductive lessons would not be 
favorably received by the teachers. However, a review of the ratings for both types of lessons 
revealed no significant differences. The response frequencies and percentages for item two in the 
lesson feedback form are given below. 



Content Rating 







a 


b 


c 


d 


a & b 


& 

> 

H 


Deductive 


46(85.2) 


2(3.7) 


1(1.9) 


3(5.6) 


2(3.7) 


c 


Inductive 


65(78.3) 


8(9.6) 


2(2.4) 


4(4.8) 


4(4.8) 


-J 















Notice that the comparative percentages in the parentheses vary only slightly. 



Evidently both types of lessons were equally well received. However, mors evidence needs 
to be gathered on the manner in which the inductive lessons were taught. 

Knowledge and Skill vs. Production. Several lessons in the first level instructor's guide 
are centered on the production of a product with the basic objective being the creation of a 
specific number of items. The remaining lessons are concerned with the development of concepts, 
knowledges, and skills related to industry. For this analysis, the general ratings of the product- 
ion lessons were compared with those of a sample of lessons whose primary intent was to dev- 
elop concepts and knowledges. A resum£ of the responses are presented below with the freq- 
uency listed first and the per cent given in the parenthesis. 
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General Rating (Item Two) 



a b c 

Production 20(52.6) 18(47.4) 0 

Concepts and 7(10) 55(78.6) 5(7.1) 

Knowledges 

The Komorgorov-Snirnov Two Sample Test gave a chi-square value that was significant beyond 
the .001 level. Category a, a rating of excellent, produced the largest divergence between the two 
types of lessons and was the basis for this significant difference. Obviously, the production 
lessons were more highly preferred. 

This result gives an idea of the teachers' preference for making articles. A further indication 
of this preference is their comments on the feedback forms. For example, one teacher wrote 
"There was not enough time in the laboratory in this unit. I was surprised that the students did 
not complain." This bias will have to be considered when new or revised materials are produced. 
Also, it will be an important factor in the design of in-service training programs. 

Summary 




d 

0 

3(4.3) 



This paper has presented a broad and detailed view of the feedback system designed and 
utilized by the American Industry Project. Lists of all of the data generated by the system have 
been omitted. Instead, summaries of the unit and lesson data were reproduced and selected 
samples of data were given for special analyses. 

In summary, the data substantiated the procedure used to determine the lesson and unit seq- 
uence. The decision to include the media narrative in the lessons was also verified. In addition, 
the detail and contents of the lessons were given a favorable rating. In fact, the complete set of 
lessons and units received a rating somewhat better than "good" on the feedback forms. 

Written comments on the forms confirm the ratings. These comments also proved to be of 
value in isolating problems. And, in several instances, they provided acceptable solutions for the 
problems identified. 

The value of the feedback system is apparent in the data and results presented in previous 
pages, information is gathered upon which to base curriculum development decisions. Baseline 
data are generated for evaluating future innovations and changes. If used properly, the feedback 
system will provide information for most if not all of the decisions encompassed in curriculum 
development at the secondary level. 

There were also some negative aspects associated with the system and the results. Failure 
to return the feedback sheets, especially the unit forms, created some problems. Specifically, 
lessons and units with a small number of ratings were difficult to evaluate. Fortunately, most 
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lessons had an adequate number of responses. Another problem uncovered was the need to mal'e 
better and more creative use of the laboratory. 

A closing word of caution is appropriate at this point. The ratings on the feedback forms 
represent the opinions of the teachers. Thus, they must be interpreted in this light. The teacher, 
instructional materials, and students comprise an instructional system. Therefore, a low rating on 
the content outline of a lesson may (ead to the decision to revise the lesson. However, a viable 
alternative decision would be to further develop the competencies of the teachers. The final 
choice will depend on the resources available and the overall objectives of the Project. 

The basic purpose of the feedback system war provide information for the decisions re- 
quired in relation to the secondary curriculum -oviiu^nent. In most instances, data from other 
sources will also be required to make these decisions. For instance, test results and responses 
to the End-of-Vear Report will have an influence on these decisions. The nature of the decision 
determines the information required. 
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At the conclusion of each unit in the Level I course, American Industry students rated the 
booklet used in the unit. The same rating instrument was utilized for all units. Twenty dimen- 
sions of concern to the curriculum specialist were placed in the Likert formate and reproduced in 
a form that could be optically scored. The items included in this instrument are reproduced in 
the summaries that follow. 

Data were collected during the first two years the booklets were used. At the conclusion 
of the two years, it was decided for reasons of time, finances, and printing deadlines that the 
booklets would not be revised during the last year of the Project. Thus, the evaluation was dis- 
continued. ^ 

Information on the validity and reliability of the instrument were collected during\the initial 
year the instrument was used. At the end of the first year the booklets had been usedV the 
research specialist examined a sample of the booklets for signs of use such as wear, smudges, and 
loose bindings. This information on useage was compared with the students' rating of the degree 
to which they read the booklets (item 18). Approximately 77% of the booklets assigned f* th£\ 
students who agreed or strongly agreed with the statement “I read most of this booklet" showed ' 
definite signs of wear. Since the "wear test" is not sensitive to all useage, it would appear that y 

the students' ratings on this item reflect their utilization of the booklets. Since this item would 
appear to be the most sensitive statement in the instrument, there would appear to be no reason 
for not believing that the responses to the other items on the rating scale accurately reflect the 
perceived quality of the booklets. 

Most of the items in the rating scale measure demension that are independent of each other. 

However, item three, "If the booklet were harder it would be more interesting," and item seven, 

"This booklet was written for someone younger than me," overlapped. The correlation (Pearson r) 
between the responses to these two items was .80. Since these two items did not measure exactly 
the same dimension, the .80 correlation would appear to indicate that the students responded 
consistently. 

The ensuing summaries are presented by grade level, year, and booklet. Also, a total sum- 
mary that encompasses the ratings for all booklets is included for each year. As noted above, the 
data pertain to the first edition of the booklets. No revised editions were produced. The fol- 
lowing comments will be based upon the two total summaries concluded from the packets of 
summary data for each year. The reader is invited to review the ratings for each booklet as 
given by specific grade levels. The number "10" in the summary table refers to grade 10 and 
above, in other words, grades 10, 11, and 12. 

Four items give a direct indication of the quality of the student booklets. Item 1 in the 
rating scale asked the respondents to indicate how interesting the booklet was. In 1966-67, 

59.2% thought that the booklets were interesting and 21.9% responded that they were not. Very 
similar results were found during the 1967-68 school year. 

Students' perceptions of the amount they had learned from the booklets are revealed in the 
responses to items 9 and 10. During 1966-67, 58.7% of the responses indicated that the student 
had learned a considerable amount of information from the booklets and 24.7% indicated that he 
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had not. The following year the students were more undecided with 50.1% agreeing that they 
had learned a lot and 21.7% disagreeing. Since this is a fairly stror statement, it would appear 
on the basis of students' perceptions that the booklets functioned effectively as far as learning 
was concerned. 

Item 10 complements item 9 in that it asks the student to indicate his prior acquaintance 
with the contents of each booklet. In each year, approximately 21% thought they knew most of 
the contents of a booklet before they read it. This figure is almost identical to the percentages 
who felt that they did not learn a lot from the booklets. This analysis does not permit the 
determination of the number of students who knew most of the contents of all booklets. 

During both school years, approximately 75% of the responses to item 12 concurred that 
the ideas in the booklets were worth knowing. Only slightly more than 8% disagreed with this 
statement. Obviously, the students thought that the material in the booklets was worth knowing. 

In summary, the booklets were favorably received by the majority of the students. However, 
as will be discussed in the ensuing paragraphs, there is room for improvement. 

As mentioned previously, the ratings refer to the first and only edition of the student book- 
lets. Certain flaws have been indicated by the students' responses on the rating scalo (NRS) and 
the participating teachers' comments. One basic problem centered on the names given to the 
characters in the booklets. The writer used the first names of the Project staff which resulted in 
the two boys being named Dick and Dan. These names caused the students to associate the 
booklets with their elementary school texts. Responses to item 7 reveal that about 36% of the 
readers felt that the booklets were written for someone younger. The simple act of changing 
names would enhance the booklets. 

Another weakness of the booklets was the limited use of illustrations. The Project's 
printing process limited the type of illustration that could be included. This limitation is re- 
flected in the students' responses to item 16. In 1966-67, 42.6% thought that the pictures 
helped to understand the booklet. During 1967-68, this dropped to 36.6%. It would appear that 
the booklets could be measureably improved with better illustrations. 

Making these two revisions should improve the students' interest in the reading materials. 

The reading level for each booklet is given in the table which follows th* - ummaries of the 
students' ratings. 
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could understand trlwl this booklet said after discussing it in class. ... 7 3.1 10.3 23.9 13.6 

.... 8 2.9 15.9 8.7 49.3 
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The pictures helped me understand this booklet 7 16.3 12.2 16.3 34.7 20.4 

8 10.8 27.5 26.7 26.7 8.3 

9 8.5 21.6 24.2 39.2 6.5 

10 10.5 16.2 33.3 29.5 10.5 
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12.9 57.3 25.0 

15.5 60.8 10.3 
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The definitions in the back of this booklet were helpful 7 18.8 18.8 25.0 25.0 12.5 

8 4.5 3.6 8.9 37.3 46.4 

9 2.4 12.1 47.6 37.9 

10 3.1 10.3 14.4 39.2 33.0 
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One of the major activities included in the first level American Industry materials was the 
student enterprise. In this activity the students designed or selected a product and determined if 
there was a market for it. If it was economically feasible to produce and market this product 
the students established a production and marketing system for it. Necessary supplies and mater* 
ials were procured for the students' enterprise. Then the students went on to produce the prod- 
uct and market it. After the marketing activities had been completed, expenses and income were 
determined and the profits or losses were identified. 

The goals of the student enterprise were to provide referents for all the concepts and the 
interactions between the concepts identified in the American Industry structure. In addition, the 
developers of the secondary curriculum wanted to provide a simulated experience in developing , 
operating, managing an enterprise. Finally, activities were sought to provide a basis for student 
interest in the American Industry course. 

Student learning materials and instructional materials for the teachers were developed to pro- 
vide the information and directions required to initiate and carry out the student enterprise act* 
ivity. These materials were designed to meet the goals described in the previous paragraphs and 
were based upon the Project's conceptual structure of the knowledges of industry. 

After the materials for the student enterprise had been developed an evaluation was designed 
to determine if the enterprise was meeting the goals set forth. An opinionnaire was constructed 
to assess the students' opinions of the enterprise activity. Table 1 contains a summary of the 
students' responses to this opinionnaire. The opinionnaire was completed immediately after the 
first unit in the first level course. This unit was culminated with a student enterprise. 

TABLE / 

SLUM ARY OF RESPONSES ON THE STUDENT ENTERPRISE: 1967-68 

American Industry Classes 



Key: SD = Strongly Disagree , D ~ Disagree . U - Uncertain , A = Agree . S/4 - Strongly 

Agree, NR = No Response 





ITEM 


SD 


D 


U 


A 


SA 


NR 


/. 


I like the student enterprise activity 


3.5 


2.8 


10.0 


52.1 


31.4 


0 . 


•) 


1 liked the group work involved in the 
student enterprise 


2. 


7.2 


14.1 


55.4 


21.1 


-> 

♦ M 


3. 


Making a profit on products we sold 
added interest to this activity 


2.6 


8.6 


13.4 


42.4 


31.6 


1.5 


/. 


Selling products produced in student 
enterprise was interesting 


2.4 


9.2 


27.9 


43.3 


16.3 


.9 


5. 


The student enterprise should be 
eliminated from this course 


52.B 


26.8 


11.6 


t. 


1. 


.6 



* Figures in the table are percentages. There w* ,rn 545 opinionnaires completed . 
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The interest value of the student enterprise activity is identified in Item 1 and 5 in Table 
1. A review of the responses to item one indicates that approximately 84.5% of the responses 
agree or strongly agree that the student enterprise activity was interesting. Only 6.3% of the 
respondents indicated that they did not like the student enterprise activity. On item five 79.6% 
indicated that the student enterprise should be retained as an activity in the course. Eight per 
cent thought that it could be eliminated. The high level of favorable response to the student 
enterprise activity would indicate that it holds a significant attraction for the students. Moreover 
comparing the responses on item one to those on item six reveals that the student enterprise 
activity is liked to a greater degree than the class in general. 

Items two, three, and four refer to the students' interest in specific parts of the student 
enterprise. These questions were stimulated to a great degree by questions referred to the pro- 
ject by industrial arts teachers and industrial arts teacher educators. For the most part they 
represent significant changes in the types of activities that would take place in the industrial 
arts laboratory. Responses to item three indicate that a large majority of the students, 73.5% 
to be exact, liked the group work involved in the student enterprise. Approximately 9% dislike 
the group work. On item four, 74% of the responders indicated that making a profit added to 
the interest of the activity. Approximately 1 1% thought that this detracted from their interest. 
The degree of agreement with item four was not as high as on the items just discussed. How- 
ever, 59.6% thought that selling the products produced in the enterprise was an interesting act- 
ivity and only 11.6% did not think that it was interesting. A fairly large number, 27.9% were 
undecided as to the interest value of this portion of the enterprise. Items seven, eight, and nine 
in Table 1 give some insight into the impact of the student enterprise activity on the students 
learning. However, the responses to these items also reflect the influence of the complete set 
of instructional materials and the quality of instruction encountered during the first unit. The 
summary of responses indicates that there has been a significant affect upon learning. For ex- 
ample, 77.2% of the students thought that they had learned a lot about industry in class up to 
this point (Item seven). One item nine, 70.4% indicated that they had a greater appreciation 
for industry after studying it in class. Only 9.1% thought that their appreciation had not incr- 
eased. Additional information on the learning outcomes resulting from the student enterprise 
activity can be observed in Table 2. The responses in this table are for the following multiple 
choice item that appeared in the Unit One Achievement Test. 

An enterprises expenses in producing a service are $100. Customers pay a 
total of $125 for this service. What is the $25.? 

A. Surplus 

B. Expense 

C. Interest 

D. Profit 









The analysis of the data in Table 2 reveals that the American Industry groups outscored the 
control groups in each of the four schools presented in the table. The control groups specified 
in Table 2 are equivalent groups. In these settings the traditional experimental paradigm was 
used. On the basis of these data, it appears that the student enterprise activity also has an 
important influence on the cognitive knowledges developed by the students, for the discussion 
of profit and its calculation was centered basically in the student enterprise activity. In other 
words, only slight mention was made of profit in the first level materials outside of the student 
enterprise activity. In summary, the data presented in this report indicate that the student 
enterprise activity presents an interesting learning experience in an effective instructional setting. 



table 2 

SUMMARY OF RESPONSES TO ITEM 30 0i\ 
THE UNIT I ACHIEVEMENT TEST 



School Choice 







A 


B 


C 


1) ( Correct ) 


Omit 


A 


A.I. 


0 


2.3 * 


0 


97.7 


0 




Control 


4.4 


8.7 


8.7 


7.9.9 


4.35 


B 


A.I. 


6.3 


2.1 


2.1 


89.6 


0 




Control 


7.3 


4.9 


2.4 


85.4 


0 


C 


A.I. 


0 


4.6 


0 


95.5 


0 




Control 


0 


9.4 


6.3 


84.4 


0 


D 


A.I. 


6.5 


3.2 


12.9 


77.4 


0 




Control 1 1.4 


17.1 


14.3 


■}7J 


') 


* Numbers in the table 


are 


percentages. 
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Introduction 



During the past four years, the American Industry Project has been comprised of particip 
ating teachers, their students and the Project staff members at Stout State Univer^ty. The part- 
icipating teachers have had an active and integral role in the development of the oject's instruc 
tional materials. In addition, these men have been responsible for the classroom implementation 
of the secondary school courses in American Industry. Hence, each participating teacher repre- 
sents a valuable source of information on the American Industry courses and the instructional 
materials used in them. 



Data Collection and Analysis 

Prior to the 1967*68 school year each participating teacher was asked to submit a formal 
written report at the end of the school year. This report detailed the accomplishments, pro- 
blems, and questions the teacher had identified during the year. This procedure produced many 
illuminating comments, but it also suffered from lack of scope and consistency. As a result, 
the information returned to the Project staff at Stout did not provide the information required 
to revise instructional materials and the structure of the courses. 

In order to alleviate these deficiences, a more structured instrument was designed for use 
as the end of the year report. The first step in designing this instrument entailed identification 
of the general area of concern associated with the American Industry courses developed for the 
secondary school. After study of the information required to develop and revise American Indus- 
try courses, six areas related to teaching these courses and the associations between the teachers 
and the Project staff appeared to be germane. These areas will be more specifically delineated 
in succeeding paragraphs. 

After the general categories were defined, several comments or statements were written for 
each category providing for teacher responses in the areas in which information was needed. An 
illustration of this is item number eight which reads "I could not understand the material in the 
Instructor's Guide." Obviously, this understanding is essential if the guide is to be of any use 
to the teache,. A large number of unfavorable responses would point out the need for changing 
the guide, or engaging in further inservice training of the teachers. In like manner, each state- 
ment directed attention to input factors over which the Project could exert some influence. 



All of the statements in the »:nd of Year Report are listed in Table 1 which is attached 
to the end of this report. The general areas identified in the first step of constructing the 
report and the items in each category are listed below: 
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Teacher Acts and 
Teaching Activities 


Content Competencies 
of the Teacher 


Utility of Project-Supplied 
Instructional Materials 


Items: 1 7 


Items : 14 


Items: 8-11 


12 


23 - 24 


15 • 22 


29 


32 


25 


36 - 37 


38 


28 


65 


47 - 49 


33 - 35 


67 


59 - 60 


39 




63-64 


42 




66 


50 

61 - 61 
68 


American Industry 


Student 


School 8t Community 


Workshops 


Competencies 


Relations 


Items: 51 • 58 


Items: 13 


Items: 27 


69 


26 


40 - 41 




30 - 31 


43 - 46 



At the conclusion of the 1967-68 school year, ee-h teacher compelted an End of Year 
Report for each Level of American Industry he had taught. The completed reports were returned 
to Stout where the data were key punched and the summary printed in Table 1 was run on the 
computer. Table 1 presents a symmary of the responses for Levels I & II. Only one teacher 
had enough students to offer a Level III class during that school year and his responses for 
Level III were not included in the summary. Five teachers taught Level II classes; thus, the 
response summary represents, for the most part, the teachers' opinions of Level I. 



The objective of this rrpor* was to give a general overview of responses on the End of 
Year Report. Therefore, responses for Levels I and II were pooled into one summary. When 
more detailed analyses are run at a later date the two sets of responses will be handled separately 
The ensuing paragraphs will give a general summary of the results in each cateogry. An invit- 
ation is extended to the reader to examine each item and assess its implications. 

Responses in the content competencies category revealed that the teachers felt some need 
for further information in each concept area. This was particularly true on the five concepts 
of transportation, management, materials, processes, and energy. 

In the other categories, the response patterns reflected favorably on the activities of the 
Project and the participating teachers. Instructional materials developed by the Project were 
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dLil mo ' 7, e r P T RrS ‘' S ° m8 h8d ,r0Ub ' e CO "’ P " i,ina *• — Leonti. 

Tons r„lZ„ k ,he lnStrUCt ° r ' Guld9 ' The *«hers' schoo, and community re- 

h nst uo," m ' C ° mPe,enCieS W6re «» par, favorably rated by 

Cted ^ ,he f ' ^0ieC, rMeived * P«sW»e rating and some valuable 
comme nts for future program, were noted by the teachers. Responses in tire teacher acts and 

teacher activities area were mixed. More will be said about this in the next section. 

Discussion 



favorable^ ^oonTTT “ ,h<> ^ ,h6 ,eaCher aCtS and lMrni " 9 “«•<««* category were 

instruction I ♦ h • OI 6mS tW °' three * thirtysix ' and thirty-seven indicate that a variety of 

rr': a r: a ; d resources wer * ^ ^ ^ », bu., nes.es jl 

world m i T C ° nCeP,S 0< indUS,rV “ 'specially valuable in linking the students' 
addition tee ar T r,farence '"COPtPCssed in American Industry (item thirty-seven). In 

reSPOrSeS to i,em SiX,V Se ' an indicate ,hat the V are no, as concerned with 
extrinsic value 0, a project as they are with its possible qualifications as a learning expedient 

Although the teachers appear to have made good use o, a variety o, learning activities 

they also appear to have overlooked two others Less than half of «ho . . . 

prs Anri nni„ ■ 1# * xners. Less than half of the group used visiting speak- 

ers. And. only half of the teachers scheduled a field trip ,0 observe an enterprise. This las, 

“"o' 1 aPP T “ kjVa . SM9h,ed ,heir *“«•»«•’ interest, as expressed in tee Uni, VII Op- 
, ... ,he ,lrst ltem " this oplnionnaire approximately eighty-two per cent of the Amer- 

Ano,r U amI ?,“ den,S reSP ° nSad '*** W ° U ' d like “ visi * a » see how it is operated, 

^uly riduce lrrm " ’1 “ i,<!m ,wa "‘V'"ine. It is conceivable that tee teacher may 

this „L1“ 0 50 0 PPPPttenities in their classes. All of the concerns noted in 

parag apn were communicated ,0 the participating teachers during the past summer. 

"TT- V" itemS ,h8t P< ’ r,ain 10 ,he amount of un< terstanding and information tee 
' ! k “! k ** e the “"“P' araas tcvssled several in which there was a deficiency. On tee basis 

Specialist Rkteard° (fhh ”*'* “I*" 1 to assist ,he Participating teachers. First, the Curriculum 

^r;, R :r “ • ° bta,ned M,era,u,e *» « he “ “"«pts indusvy an d d is . 

workshon ° a* “ CherS ' Second, ^ most 0 f the morning session of the September on-campus 

Industrial TechnT 1 * n° * le ctiire-discussion of tea management concept by a member of the 
industrial Technology Department at Stout. 

There *"» • favorable response ,0 all items in the student competencies category except for 
W '* h ^ tee students could assume in the stiiden, enterprise 

ronV oatt l , . Ieva ' C0UrSa ' Tha fU " impa,!, ° f ,his teapotte Is received when the 

i,rrrr V. ,wen,y - nine ™ considered in conjunction with it. On 

m twenty-nine, eighteen teachers agreed that they did a lot of work ,0 assure the success of 

tee stede nt enterpr ise Later, on item sixty-, we. fourteen agreed tea, they do whatever is nec 
estery ,0 successfully complete a product or activity. Considering these responses in aggregate 

make misttkes in tee learning activities. Unfortunately, this undue concern may deter the use o, 
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problem-solving activities or make them po barren as to be useless. This concern was also ex- 
pressed to the participating teachers. 

On the basis of the responses in the school and community relations category the teachers 
seem to be doing an effective job of public relations with their peers, administrators, and com- 
munity. The Project's workshops also received a favorable response. In addition, the teachers' 
comments on item sixty-nine were considered in scheduling 1968-69 workshops. Agendas were 
lengthened and more outside speakers were scheduled. 

With but two exceptions, the response patterns on the items related to the utility of the 
Project's instructional materials were favorable. The teachers indicated that more test items 
should be included in the guides and that provision should be made for more flexibility in teach- 
ing the courses and using the guides. Time was not available to develop more test items. How- 
ever, provisions were made for each teacher to modify his course within the bounds of the 
American Industry rationale. 

Review of a sub-set of the items in this category which drew a favorable response will 
further illustrate the value of the information provided by this opinionnaire. Items eight, sixteen, 
seventeen, thirty-three, thirty-nine and sixty-two comprise this sub-set. Responses to these items 
reflect the teachers' positive opinions on the quality of the format and contents of the lessons 
in the guide and the degree of improvement made in the guide. These data suggest that the 
Project can continue to employ the lesson format, the style of writing used in the lessons, and 
the level of detail utilized in the lessons. 



Summary 



Use of a structured report form for the American Industry teachers provided the Project 
with a wealth of drjta in several areas related to the development of instructional material'. and 
implementation of secondary school courses in American Industry. Opinions were obtained from 
all teachers on each of the items; thus, decisions could be made on the basis of the opinion of 
the entire group. The written report originally used usually gave a variety of responses with any 
one response being listed by only a few teachers. 

Overall, the responses were favorable. Some actual and potential problems were identified 
and the participating teachers were informed of these. 
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TAM A'] I 

SUMMARY OF RESPONSES ON THE 1967-68 
ENI) OF YEAR REPORT FOR LEVELS I * II 



Sl> - Strongly Disagree l) - 

l-l neertain - 

N.l - Strongly :S& ree NR - 

1. I found that students could apply their 

knowledge in science and math in my 
American Industry classes 

2. I used buzz groups during the year to 

discuss various questions 

3. I used brainstorming in class to come 

up with ideas 

■1. Visit:. tg speakers talked to my American 
Industry classes 

5 . We went on field trips to see enterprises 

6. I frequently assigned reading in the 

student booklets as homework 

7. Students were not allowed to take their 
student booklets out of the school building 

8. I could not understand the material in the 

Instructor's Guide 

9. Effective use of the overhead transparencies 
requires too much narration by me ... . 

10. Use of overhead transparencies retains the 

students' attention better than straight 
lectures 

11. I have more instructional resources to 
use in my inducainal arts classes . . . 

12. I used small enterprises th/oughtout the 
year to help provide meaningful activity 

13. In Unit VII, the students were able to 

carry out all of the achitivites of industry 
with very little help from me 

14. I need more information in the transportation 

concept area 

15. I would not be able to teach the course 
without the Instructor's Guide .... 

16. This year's guide is much better than 

last year's 

17. The format of the guide is easy to follow 



Disagree 

Agree 

No Response 



SD 

0 

I 



8 

7 



15 



II 



11 



SD 

0 

1 



D 

3 

2 

2 

4 
3 

3 

1 

4 
4 

0 

8 

4 

7 

4 

2 

D 

0 

0 



u 

4 

I 

0 

0 

0 

I 

•4 

3 

4 

3 

5 

1 

5 

7 

4 

U 

3 

1 



A 

12 

9 

9 

3 

6 

3 

0 

2 

I 

5 
0 

7 

3 

6 

8 
A 
7 

II 



SA 

I 

7 

9 

5 

4 

7 

0 

0 

0 

12 

1 

7 

1 

1 

4 

SA 

9 

7 



NR 

I 

I 

9 

I 

I 

I 

I 

I 

1 

I 

I 

1 

2 

1 

/ 

NR 

2 
1 
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Unit introductions in the guide are of 

it t tit help . 

More Irst items should be included . . . 

7 he complete tests included in Units I 
& III ii ere effective in my classes . . . 

The narrative in the lessons did not give 

enough content 

I could tell by the narrative in the lessons 
when I teas to use discussion or other 

student involvement 

/ have sufficient background to teach the 

finance lessons 

/ need more information in the property 

concept area 

The Instructor's Guide did not provide 

for discussing local enterprises 

Most students could identify t ie concepts 
used by lies as he managed Pronto PreFabs 
l/v school does not like to have, students 

sell products produced in school 

The student enterprise activity works 

effectively in my class 

/ had to do a lot of work to assure that 
my student enterprise would be a success 
My .4 merit an Industry students improved 
their conceptual style of reasoning .... 
Safe working habits were hard to develop 
l have enough background in the research 
concept area to teach it effectively .... 
The lessons did not go into enough depth 
The instructional media were effectively 

correlated with the lessons 

The instructional media are not an 
important part of the materials supplied 
by the American Industry Project .... 

/ found that newspapers and periodicals, 
such as Time , had a lot of useful infor- 
mation for the American Industry course 
Discussion of local enterprises helped 

to bring out the concepts 

/ need more information in the pro- 
curement concept area 



"'w 



SI) 

39. The format of the lessons is functional O 

40. My colleagues in the industrial arts 
department share a favorable attitude 

towurd my American Industry courses 3 

II. The administrators in my school feel 
that my American Industry courses are 
an important part of my school's curr- 
iculum 0 

42. I had trouble completing the course on 

schedule 0 

43. I have received requests to make a 

presentation on American Industry ... 5 

44. I had several visitors in my American 

Industry classes this year 3 

45. Other teachers in my school system would 

like to teach American Industry .... 5 

SD 



16. If there were other teachers of American 

Industry in my school system , / believe the 
administration would initiate all three levels 
of 1 merican Industry in the school curr- 
iculum 3 

17. I need more information in the communi- 
cation concept area 3 • 

18. I need more information in the relation- 
ship concept area 2 

49. I need more information in the marketing 

concept area 2 

50. I had more instructional material to use 

this year as compared with last year . 0 

51. The Workshop helped to answer some of 

my questions I 

52. The Horkshops are a valuable part of my 
portu ipation in the American Industry 

Project / 

53. The most important part of the Work- 
shops is the chance to talk informally 

with other members of the Project . . 0 

51. I appreciate the summary of events given 
by the Project Co-directors at the Work- 
shops at Stout 0 

.55. More lime on the Workshops agenda should 
be given to the presentation of information 
and content in the concept areas . . I 



I) 

0 



3 



l 

I 

5 

0 

5 

D 



1 
7 
7 

10 

2 
2 

0 



2 



l 

0 



U A S I 

I 13 5 

7 6 I 

4 0 9 

l 7 II 

I 4 5 

0 7 10 

7 2 0 

U A SA 



8 3 5 

l 7 2 

3 7 I 

l 6 1 

l 9 8 

1 9 7 

1 6 9 

I 13 4 

Oil 8 

7 6 6 



Mi 



l 



I 

I 

I 

1 

2 

NR 



I 

I 

l 

I 

l 

I 

I 



/ 



/ 
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SI) 


n 


U 


1 


56. 


1 he Workshop programs were too tightly 












scheduled 


5 


i 


1 


3 


.17. 


1 rea workshops are helpful . . . 


6 


0 


*) 


t 


.»«. 


1/ each workshop one of the participating 












teachers should present a summary of what 












materials have worked effectively for him 


6 


i 


5 


10 


59. 


/ have enough information in the resources 












concept area .... 


0 


7 


1 


7 






SD 


D 


U 


A 


66. 


/ need more information in the management 












concept area 


4 


4 


2 


9 


6/. 


Overhead transparencies were more effective 












than the slide and filmstrip series 


6 


7 


10 


3 


6l\ 


Having the student activity sheets printed to 












facilitate duplication was a significant improve- 












ment over last year .... 


6 


0 


3 


8 


63. 


/ need more information in the materials 












concept area 


3 


3 


2 


9 


64. 


/ need more information in the processes 












concept area 


5 


4 


0 


11 






SD 


D 


U 


A 


65. 


/ do whatever is necessary to assure that a 












product or activity will be successfully 












completed by my students 


0 


5 


0 


10 


66. 


I need more information in the energy 












ncept area 




3 


2 


11 


67. 


Products produced in a student enterprise 












must he complex enough to give an observer 












a good impression 


5 


11 


3 


I 


68. 


The Instructor's Guide could be improved ♦ 


0 


1 


1 


11 



List ways it could. 6e improved: 



S I MR 

I I 

10 2 



1 / 

5 / 

SA MR 

I I 

0 I 



9 / 

3 / 

0 / 

iVl MR 



4 2 

I I 



0 / 

5 3 



69. The workshops could he improved 
List ways they could 6c improved: 



0 2 2 9 i 4 
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Introdvction 



One of the hypothesized products of the study of American Industry in the secondary 
school was the development of a conceptual understanding oi jstry. In addition, it was 
thought that the students would acquire additional factual information related to industry. The 
American Industry Achievement Test described in the ensuing pages was designed to measure 
these products. It is important to note that this was only one of several measures employed to 
assess the end products of the study of American Industry. An essay test, several opinionnaires, 
and a follow up study were also administered to determine the effects of the study of American 
Industry. 

The ensuing section of this paper details the rationale used in designing the American In- 
dustry Achievement Test. Succeeding sections describe the design of the instrument and the 
validity and reliability measures employed in its development and refinement. Analysis of the 
results of the testing program are presented in the last section. 

Rationale 



Measuring knowledges and conceptual understandings involves an indirect system of measure- 
ment. In other words the researcher cannot directly observe or measure the concept, instead this 
must be done indirectly throught the presentation of test stimuli and the recording of responses. 
Only through proper structuring of the test stimuli can the use of concepts be evoked internally. 
After the test stimuli have been presented to the students, the evaluator must estimate from their 
responses the degree to which a specific concept has been utilized in responding. This set of 
circutu!»tar>? 0 '. had two implications for the construction and interpretation of the results of the 
American Industry Achievement Test. First, the nature of conceptual thinking and responding 
had to be carefully delineated. This provided the framework for developing the test items and 
the scoring system for the responses. Second, it meant that the test data would not exactly 
reflect the degree to which the responders have attained the given concepts. For, the fact that 
an indirect system of measurement had been employed suggested that there might be some errors 
or lack of precision in the test data. However, this was not as constraining a factor as might 
first be surmised. All measures contain some error and thus the task became one of minimizing 
these errors and developing test items that were as valid as possible under the constraints in- 
volved in this project. 

Some basic concepts in measurement have been discussed briefly in the previous paragraphs. 
Another factor in the development of the achievement test was the nature of the concepts and 
ideas to be measured. The guidelines for this aspect of the test development came from the 
American Industry Project's conceptual structure of the knowledges of industry. Inherent in this 
structure were a selected set of concepts and the interrelationships between these concepts. 

(See Appendix D) In addition to this structure the Project's definition of industry also estab- 
lished some parameters around the area of knowledge to be assessed. Since the structure and 
the concepts had been specifically defined, a sound basis had been provided for writing a valid 
test. In fact, these materials greatly facilitated the development of the evaluation instruments 
and procedures. 
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The basic elements to be evaluated as products of the study of American Industry in the 
secondary school were the concepts which appeared in the structure and the interrelationships 
among these concepts. Thus, before the test instrument could be written, it was necessary to 
investigate the role of concepts in interpreting and responding to stimuli. 

A variety of definitions of concepts have appeared in the pyschological literature. For ex- 
ample, Bruner (1957) viewed them as categories that are developed through experience. Utilizat- 
ion of a person's concepts, according to Bruner, depends upon the degree to which the stimuli 
agrees with the experiences encompassed in a given category. Categories tend to be homogeneous 
in nature and only when experiences included in a category become too heterogeneous does the 
category subdivide into new categories. 

Staats (1961) has explained the acquisition of concepts as an S-R type of learning. Accord- 
ing to Staats, concept labels derive meaning from the learner's experiences. Thus, experiences 
determine the composition and the level of sophistication of a given concept. Bruner, Goodnow. 
and Austin (1956, p. 244) defined a concept as "... the network ol inferences that are or may 
be set into play by an act of categorization." These authors also felt that the possession of a 
given concept sensitized the person to took for the attributes of this concept in the stimuli en- 
countered in the real world. Carner and Sheldon (1954-55, p, 226) concluded from their research 
that a concept could be viewed as a psychological construct which developed from experience 
and has some utility in the individual's thinking or responding. Burton, Kimball and Wing (1960, 
p. 154) suggested that a concept is detached from the central stimuli that give rise to it. 

Although a variety of definitions for the term concept were encountered in the research 
literature there was a certain degree of similarity in the definitions. Most writers agreed that a 
concept permitted a person to judge a set of dissimilar objects of phenomena as equivalent on 
the basis of one or more defining attributes. For example, Bruner talked about the act of cat- 
egorization and Burton, Kimball and Wing discussed the fact that a concept was detached from 
the specific stimuli that gave rise to it. Thus, one critical attribute of conceptual responding 
would be the ability to perceive novel stimuli as appropriate to a specific concept. 

As concepts are learned they tend to develop from the physical aspects of the events and 
progress to a symbolic abstraction. Bruner (1964) has summarized man's attempts to compre- 
hend the world's phenomena as a process of moving from seeing them in terms of their physical 
properties to a comprehension of the symbolic commonalities involved in the set of events. 
Therefore, it would appear that the saliency of the student's response may reflect the outcomes 
of the study of American Industry. 

The taxonomy for the cognitive domain provided additional insights into the hierarchy of 
cognitive skills required to respond conceptually. On tho basis of the previous discussion, con- 
ceptual behavior would fall within the third category of the cognitive domain, "application." 

This level in the taxonomy deals with the ability to apply knowledges previously acquired in new 
situations. This is parallel to the nature of the conceptual response. If the hierarchical struct- 
ure of the cognitive taxonomy is valid, conceptual responses would also require that the responder 
have the appropriate first and second level knowledges. This implies that there are certain facts, 
procedures, and interpretations that will have to be utilized before one can move to conceptual 



responding. Thus, if it is found that a learqer cannot respond appropriately to a conceptual test 
item the evaluator will have to determine whether the student lacks all of the facts or know- 
ledges required or if he is unable to make the transition from the interpretation level to appli- 

MtiAn * - 



A review of the information presented in the previous paragraphs identified several prin- 
ciples that are to be used in the development of the Achievement Test. First, items that were 
to test concepts had to involve a stimulus that would be capable of evoking the concept. Second, 
the stimulus had to be appropriate to the American Industry Project's conceptual structure of the 
knowledges of industry. Third, the test items had to contain stimuli that were novel to the 
responder. Fourth, test stimuli had to display various levels of saliency. Fifth, in order to dia- 
gnose problems in the attainment of concepts, items should be included in the test to determine 
at what level the student had acquired the knowledges of industry. This meant that the test 
should include some knowledge and interpretation items as well as the conceptual items. 

Designing the Achievement Teat 

A review of the amount of time required to complete the entire set of evaluation instru- 
ments used by the project indicated that the Achievement Test should be completed in one 
class period. This time limit also made it feuible to have a Project staff member administer 
the test. Since a relatively large number of students were encompassed in the evaluation the 
multiple choice format provided almost efficient means for gathering a fairly large volume of data 
on the cognitive outcomes of the study of American Industry. 



The large number of concepts and interrelationships incorporated in the structure made it 
impossible to measure the attainment of each of them. Thus it was decided to measure a sam 
pie from the total set of knowledges and concepts that should have been produced by the study 
of American Industry. At least one item was included on each of 13 concepts on the ball of 
the structure and the 5 major concepts in the environmental ring. However, the number of 
interrelationships was very large and only a sample of these could be included. In addition, in 
only 6 conceptual areas was there the opportunity to include items based on levels one and two 
in the cognitive taxonomy. Table 1 identified the content of each of the 64 items in the 
Achievement Test. Content was divided into main concepts and secondary concepts. Main con- 
cepts involved the primary knowledges required to answer the question correctly. In other words 
the stem and the correct response for item 1 were based on the materials concept. The second- ' 
ary concepts were incorporated in the distractors in this itam. The knowledge of an interrelat- 
ionship was evaluated in the items where there is more than on. concept listed under main 
concepts, such as Item 44. 61. 52. and 57. Also the knowledge of interrelationships was involved 
to some degree in the ability to distinguish between the secondary concepts involved in the item 
distractors. Table 2 gives a summary of the number of times each concept appears in primary 
and secondary roles in the test. Concepts on the ball of tha structure were used in a primary 
role in from two to as many as seven items. Management, production, and research were three 
conceptual areas in which additional item, at levels on. and two in the cognitive taxonomy were 
included in the Achievement Test. This explains their higher frequency of appearance in a pri- 
mary role. Use of the concepts in the secondary role varied from a low of one to a high of 



J-3 



268 




twelve. Since the environment ring contained five major concepts, the twelve listed for this area 
is somewhat deceiving and it actually represents slightly more than two inclusions per concept in 
the environmental ring. Finance was also used twelve times in a secondary role. This was 
mainly caused by the fact that money or financial considerations are associated to some degree 
with many of the activities in industry. Similarly, communication is involved in relatively large 
number of times in a secondary role. There appears to be a satisfactory distribution of items 
by concept and by role. As will be noted later, the secondary role has a greater importance 
than just providing distractors for items since appropriately written distractors can identify some 
of the misconceptions of the responders. 

The general format for the conceptual multiple choice items included in the test was a 
stem which contained unique problem situation related to industry and a correct choice based 
on the Project's conceptual structure. The achievement test was designed to provide feedback 
information for the refinement and revision of the instructiona materials as well as to be a 
gauge of the final attainment of the students. Thus, a number of the items were designed to 
identify through the student's choice the basis for his responding incorrectly. If a wrong answer 
was selected the Researcher and Curriculum Specialist were able to identify the reason for his 
misconception. For example, one of the terms included in the test was the following: 

Why would mahogany panelling be used in the office of a president of an enterprise? 

A. To reduce building costs 

* B. To communicate his status 

C. To improve the insulation of the office 

D. To strengthen the office walls 

This question was a test of the student's comprehension of the relationship between communicat- 
ion and some of the attributes of material. The correct answer required that the student 
comprehend the relationship between communication and materials. If the student focused on 
the salient physical qualities of materials his choice would be D and this could then be fed back 
into the curriculum revision process. 

Two conceptual items requiring pure conceptual thinking isolated from content were included 
to ascertain if the conceptual study of American Industry had any carryover effect on conceptual 
thinking in general. These items were related to industry but the knowledge of the concepts of 
industry could not be directly applied to obtain the correct answers for these questions. 



A formal set of test adminstration procedures were developed by the Research Specialist 
and used for each administration of the Achievement Test. The administration procedures con- 



the second section included the directions to be read to the students and the instructions to the 
test administrator for the observation and control activities to be employed during the test period. 



Test Administration Procedures 



tained two sections. The first dealt with the preparation required before the class arrived and 
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These instructions were rather detailed and were piloted by the Research Specialist during several of 
the pretesting sessions in the initial year of the evaluation period. Slight revisions were made and 
then additional copies of the instructions were printed and placed in 8% x 11 binders. The Research 
Specialist, other Project staff members, and guidance counselors in the participating schools used 
these instructions during the three years the achievement test was administered. No specific weak- 
nesses were noted in these procedures. 

Since the testing period consisted of two timed sections in the Achievement Test and a 
separate essay section, a timing sheet was designed to assist the test administrator in keeping on 
schedule. This sheet also included a place to note any problems that occurred, during the test per- 
iod and allowed for feedback of any oddities that might have transpired in the testing. Review of 
these forms revealed no events that would invalidate the test data. 

During the three years the Achievement Test was used, several different procedures were 
utilized to administer it to the American Industry and control group students. During the first 
year, the Research Specialist and other Project staff members administered all of the tests. After 
the first year, a reduction in the Project's budget required that the travel of the staff be restricted. 
As a result the Research Specialist had to turn to the guidance counselors and participating teachers 
to administer the test. Some question might be raised concerning the use of the participating teach- 
ers to administer the test. However, the data were available from the previous year and comparisons 
between the first year and the last two years indicated that there were no irregularities. In addition, 
the tests were sent to the principal of the schools involved about two weeks before they were to be 
administered. On the day of the test the participating teacher would obtain the test materials from 
his principal, administer them, and return the completed answer sheets and the test booklets to the 
principal before the close of the school day. This procedure worked satsifactorily and did not ap- 
pear to result in teaching of a test. 



Content Validity 

In the section on designing the Achievement Test, the basis for sampling the concepts and 
interrelationships to be measured was defined, in addition. Tables 1 and 2 present summary data on 
the concept areas tested by each of the Achievement Test items and the distribution of test items 
by concept. The distribution of test items appears to be appropriate. 

As additional consideration in the area of content validity is the type of knowledge measured 
by the test items. In other words, the test items must require conceptual behavior to define the 
correct response in order to be valid. As noted in the section describing the design of the Achieve- 
ment Test, Bloom's Taxonomy for the Cognitive Domain was used to identify the level at which 
each Achievement Test item was functioning. Table 3 presents the taxonomy level of each of the 
test items in the Achievement Test. A review of Table 3 indicates that an adequate number of 
level three items or higher in the taxonomy were present to assess the conceptual knowledges of 
the students. Care was taken in writing the test items to use a vocabulary and sentence structure 
appropriate to eighth grade students. 



Validity and Reliability 
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TAIil.E I 

Content Analysis of American Industry 
Achievement Test 



Section 1: 



Item 


Main Concept 


Secondary Concepts 


l 


Material 


Processes , Procurement , Production. 


•> 


Production 


Finance. Resources. 


3 


Communication 


(f«'y :<) 


1 


Property 


Materials , Procurement. 


5 


Processes 


Conceptual. 


6 


Transportation 


Marketing , Finance , Competition. 


4 


Production 


Finance , Resources , Management. 


8 


Productivity 


Compeition. 


9 


Knowledge of 1./. Structure 




10 


Finance 




11 


Productivity 


Finance , Resources , Management (plan) 


12 


Environment 




13 


Relationships 




14 


Management 


Communication , Production. 


15 


Competition 


Pro fit(Fi nance). 


16 


Marketing 


Research , Need. Competition. 


17 


Research 


Finance , Communication. 


18 


Management 


Relations to Entire Enterprise-Production , 
Procurement , Marketing. 


19 


Procurement 


Materials , Transportation , Need. 


20 


Energy 


Relationships. 


21 


Relationships 


Research 


22 


Production 




23 


Research 


Marketing. 


24 


Marketing 


Co mmunica tion s. 


25 


Procurement 


Materials. 


26 


Processes 


Com munication. 


o - 

t 


Materials 


Finance. 


28 


Energy 


Conceptual. 


29 


Transportation 


Energy. 


30 


Management 




31 


Transportation 


Energy. 


32 


Conceptual 


Relationships , Procurement. 


Section II: 


33 


Transportation 


Finance 


34 


Relationships 




35 


Research 
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✓ 



T 1 ItEE I (Continued) 



hem 


Main Concept 


Sec ondary Co nee p is 


:t(> 


Relationships 


Materials , Communication , Finance. 


37 


Procurement 


Transportation , Finance 


38 


Definition of Industry 


Productivity 


39 


Finance 




w 


Management 


Marketing 


it 


Future 


Productivity , Environment , Definition of A. 1. 


12 


Conceptual 




ts 


Production 


Finance. 


it 


Processes-Ma terials 


Procurement , Transportation. 


to 


Research 




■16 


Processes 


Finance , Energy. 


17 


Management 


Relationships , Finance. 


18 


Finance 




19 


Production 


Resources 


50 


Environment 




51 


Managemen t-Production 


Communication. 


52 


Managem en t-Produc lion 


Communication. 


52 


Contmunication 




5i 


Property 


Materials. 


55 


Research 




56 


Materials 


Definition of industry. 


57 


Fu ture-Hii'tory 


Relationships. 


58 


Marketing 


Communication. 


59 


Processes 


Communication, Environment. 


60 


Energy 


Transportation, Finance, Environment. 


61 


Property 


Processes, Production, Management Environment. 


62 


Energy 




63 


Finance 




64 


Marketing 





K7Z 
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TABLE 2 



Frequency of Appearanee of ('ow e pis 
in l ho 1 chievement lost 
See lion I Soot ion II 

Coin opt I ■ :i2 33 - (A Total 





Primary 


Secondary 


Primary 


Secondary 


Primary 


Seconder 


Communication 


1 


5 


1 


5 


2 


10 


Transportation 


2 


3 


2 


3 


4 


6 


Finance 


3 


10 


1 


2 


1 


12 


Property 


l 


1 


2 


0 


3 


1 


Research 


3 


l 


3 


0 


6 


l 


Procurement 


2 


4 


1 


1 


3 


S 


Relationships 


3 


2 


1 


1 


4 


3 


Marketing 


2 


2 


2 


2 


4 


l 


Management 


6 


3 


1 


1 


7 


l 


Production 


6 


2 


1 


2 


7 


4 


Materials 


2 


3 


2 


2 


4 


5 


Processes 


3 


1 


2 


2 


5 


3 


Energy 


1 


1 


3 


2 


4 


3 


Environment 


3 


9 




3 


3 


12 


Conceptual (pure) 


l 


1 


1 


1 


2 


2 


Productivity 


2 


2 






2 


«> 


Definition of n ./. 


1 


4 






1 


l 


Structure 


1 








1 




Future 


1 








1 
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TABLE 3 

Summary of the Cognitive Taxonomy Levels and Concept 
Areas in the American Industry Achievement Test 




Property 
Research 
Procurement 
Relationships 
Marketing 
Management 
Production 
Materials 
Processes 
Energy 
Environment 
Conceptual 
Productivity 
Definition A.I. 
Structure 
Future 



2 

1 

1 



1 

2 



2 

1 



2 

2 

2 

4 

2 

3 

2 



4 

4 

3 



2 

l 

I 

1 



45 
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TOTAL 



8 



7 



3 



I 




The contents of the Achievement Test were reviewed by other members on the Project and 
were judged to be valid. 

Reliability 

Table 4 reports the low median, and high reliabilities obtained from the administration of the 
Achievement Test. These reliabilities were calculated by means of Hoyt's ANOVA. Median reliabil- 
ities for grades 8 and above exceed .80. While not a high as the reliabilities of most standarized 
tests, the reliability of the Achievement Test was adec jate for the purposes of the evaluation. Also, 
it should be noted that the problem type items included in the test tend to reduce test reliability. 

Median reliabilities increased with the age of the responders and the range of reliabilities 
obtained decreased with age. Both of these phenomena would be expected if the Achievement 
Test was accurately measuring the student's conceptual knowledge of kndustry. 

In summary, the Achievement Test was found to be content valid and reliable. The relisbil 
ities obtained for grade 7 were low; however, the test was designed for grade 8 and above. Thus, 
these results were not unexpected for grade 7. 

Reactivity of Presting 



Pretests were administered whenever students could not be randomly assigned to gooups. To 
check the reactivity of pretesting, a Solomon four-group design was used in two schools during the 
1965-66 school year. In these schools two pretest instruments were employed. One was the Project's 
Achievement Test. The second test was based on traditional industrial arts content. Prior to the 
test period the two test forms were collated in alternating order * Achievement Test, traditional 
test. Achievement Test, etc. • into one stack. Tests were distributed in consecutive order from this 
stack at the start of the test period. 

Analysis of the post-test scores for the students in American Industry and the control groups 
revealed no statistically significant increase in test scores resulting from exposure to the pretest. The 
outcomes of the one-way analysis of variance comparisons are given in Table 5. (Ability strata were 
not included in the analysis because of the small and unequal numbers in each of the resulting cells.) 
None of the F values were significant, thus, pretesting did not cause any significant change in per- 
formance on the post-test. On the basis of these results, it was concluded that pretesting could be 
conducted without the placebo. Administering the Achievement Test to all students in the control 
and American Industry groups at the beginning of the course provided a larger N for use in analysis 
of covariance. 

Results of Comparisons of Achievement Test Performance 

Analyses of the achievement results are presented for each school in the following pages. 

A summary of these results is given in Table 6 which follows immediately after this page. This 
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TABLE 4 



Grade 


Reliability of 

1 <■ // it 


the American In las try 
•vein ml Test 

Reliability 




Level 


Number 


Low 


Median 


High 


7 


7 


.66 


.68 


.70 


8 


24 


.59 


Ml 


.90 


9 


14 


.58 


.84 


.88 


10. //, & 12 


28 


.76 


.85 


.91 
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Solomon Four ■ Croup Analyst's 



School L 



American Industry 



Source 


d.f. 


SS 


ms 


F 




Pretesting 


1 


11.49 


11.49 


.13 


N.S. 


Error 


24 


2025.12 


84.38 






TOTAL 


25 


2036.61 








Control Groups 












Snnrre 


jLf 


SS 


MS 


F 




Pretes* 'ig 


l 


6.53 


6.53 


.08 


N:s. 


Error 


28 


2324.67 


83.02 






TOTAL 


29 


2331.20 









\merican Industry 
Source d.f. 


School M 

SS MS 


F 




Pretesting 


1 


.83 


.83 


.019 


N.S. 


Error 


18 


782.92 


43.49 






TOTAL 


19 


783.75 








Control Group 










Source 


d.f. 


SS 


MS 


F 




Pretesting 


1 


39.67 


39.67 


.75 


\.S. 


Error 


18 


948.13 


52.67 






TOTAL 


19 


948.8 
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Summary of Com fxtri sons 
on 1./. Achievement Test J'or Level I 
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disordinal inh rat 'lion 
significant gum scores 



table summarizes the outcomes of the comparisons made between American Industry (experimental) 
and control groups on the Project's Achievement Test. The difference in performance between those 
in semester and year programs and the difference associated the degree to which the instructor's 
guide was followed are indicated in the table. Programs which were a year in length produced 
proportionately more significant differences than those which were a semester long. None of the 
teachers who did not follow the guide had American Industry classes that performed signifcantly 
better than control groups on the achievement test. During 1967-68, the impacts of major revisions 
in the instructional materials can be observed. For this year the instructor's guide was completely 
revised and new media was developed. The long range results, however, can be seen in the improved 
performance in 1968-69. 

The results of comparisons within school are presented after Table 6. Each school is 
identified by an upper case letter. These letter labels were randomly assigned to the schools. The 
year, length of program, and level of the American Industry course(s) involved are also listed. In 
the analysis of variance tables the ability component is based on high, middle, and low groupings 
with + Vs standard deviation (from the test norms) bounding the middle group. When only two 
ability groups are included in a table (1 d.f. for ability) the two levels represented are usually the 
high and middle groups. Review of the ability data presented in the concluding section of this 
reoort will indicate whether high and middle or low and middle ability groups are encompased in 
the analysis. Special analyses are footnoted in the tables. 



COMPARISON or WIFRICAN INDUSTRY ANO CONTROL 



0 



v: 

c— 



l 



v: 



3 S 

O 



fc 

*4 

2 

5 

«* 

Ov 

3 

* 



^ *1 a 

131 











k. 

0 

£> 


«e 

S-SP 






£ 


-rS: 
















aocA, 






a. 


V e ' 








cc Os oc 






* 6 . 


l^MN 








^5 n: 


cc 














«© ^ « 


«o Jf* 






evi ws 


CV 5 N» 




CO 


_ ♦ ♦ ♦ 
00 «C Vi 


<55 u$ 




«• 


N •» >0 


o *0 




Jg 


<*ti vO 


50 




^ ♦ 
« 


*1 Oj 


tO lo 


CO 


Q 




to 


sO 




*0 

to -*■ Sfc 


Os cyK 

sO Os 


•*« 

co 


CO 

c /5 


SO cc 

JO — « 

vo *■■• 


S to 

O vo 
to Os 


o\ 

$ 










c 

0 




$ «? 
§ | 




<*•» 




*5 J 




■S 

i 


*s 

e 

41 


j| 




0 ) 

8 

§ 


S* 
a ?» 7 S 
c-o^ 


Ofc e* 

§ 1 
S £ 

^ B 


a 

o 






bm 



fe 

CO 

oC 

ci 

$ 



Favors 






ft. 


v5 




• 


s? 






50 




ft. 


N* 

• 






Os Csj 




CO 


cc oo 




4 


99 «£ 

CO Tf» 








e\J 


ft: 


to 


to 


cj 








00 tSi 


io 


rX 


K Os 




CO 

.j 


«n <o> 


cd 


co 


tN On 


sO 






to 




<N 




5 

o 






•im 

** 






•3 






£ 


>*■* 






s 




0- 


1 , 




k 


a a 




a 

CO 


X t t 

ft* ft] 


,o 

c— • 



J-15 

280 



0) K. 

e | 

-a 5 n 



O 

ERIC 



0 



2 








V. 




f" 5 ?? 

3 * 

Si* 


0 

& 

3 




<* 










5 

*<* 


? 












S* 






<*» 






c* 


•W 




a 




















Li 


to Li 




SO 


Li Li 


> 

V. 

• 


!- 


K?e 




^ li L L 




to 




*— 


i *. 
1 V 


> 




1 

/ 

/ 

1 


?S 




Li 

Ln ^ Li 


.* 




• -■ • • 






S «* 2c 25 






C\ 

i • * • 


■n 




r* Ll — 
Os Li to 






A 


-o 




• » • i 


"0 




5i 






% 


35 






& 

<2 




cr* 


0 






2 




On 






3 

3 



~i 




S 

CL 

* 



vC 



TOTAL 


Treatments 

Ability 

T X A 

Error \ 


Source Variation 


1501.42 


La 

£>C£ -4 
4LSO Os 

• • • * 

NONO On No 
OOOO *-J 30 


?> 

S'. 


to 


NO 


P 


^4 


*. 






La 

4 lOo «sj 4L 


K 




r*io r- ^ 


P 




4* No On tso 






N 00 O <sj Co 








•*3 




1 sc *— 






Li *SJ to 






. > A* 






:»2f« 


\j 




\ • 

a; 


35 




. «• . 


Q 

r; 




s- 


S 






2 



s ■ 

SI 1 

** 

3 « 

a 

"*l 



J-16 





COMPARISON OF AMERICAS INDLSTRY AND CONTROL 
CROUP STUDENTS ON THE ACIHEIEMENT TEST 



School H 

Level I - 9th tirade 




v» 










«? 

>• 




-c 








* 


1 












1 
















•/J 


iri • 










• 










V 






C*. 




01 

.00 


















0 


© ^ 


»■■■ 






0 


ITS 


Os 




C/5 


vO 


• V 
•*- <0> 


• 






00 


*1 
























« 












— 
















*i 






£ i 


?v* 




yi 


sC 


• _• 


oc 


*a 






CO ^ 




05 








uo 


r >. 






























5-» 












ss 

0 








e^* 


• •« 










•S 












42 










e 

A \ 








0) 


w 

g 












** ' 


V. 




w 

2 


J 


g* 


J 




A 


?•* 


"T ?-* 







S 



% 

a s 

1 j 

CO <*> 

« °P 

Os r^ 

• £ 



£ 






O 


~c 




» 






£ 


a: 






^ - 




Q. 


a: ® . . 






V 






25 Os ® 






^ Os ~" , 

• . • * 






00 




vj 


^ ^ 






Q> ^ e»0 ^ 






• • • _• 
--»• e*i co 






O e-* 




t* 






Cj 


1 

1 

1 

20 


S3 


c/i 


2 S' *-* ® 

O © f*5 »0 


<£ 


<*< 


• • • <* 
**m m* fiV} Os 






O v© 

>© 


t^ 

Os 












rr 






£ 






p 








« 






.0 
• "Ml 






4» 

•s 






ft* 

JS 


*5 

** 






C 






%> 




* 

k 

§ 


l£~ . 
J 2 * § 




<A 


&-. ^ &». t*j 





& 



% 1 



J-17 



28? 



I 'n weigh ted means method used for unequal frequencies . 
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scored significantly better than the middle ability in the control group which had 4./. during the first semester . I he status 

in the comparison between high ability groups was reversed L iSote , /fte tendency toward significant differences between treatments 
was cau*?d bv the interaction. 



^ [fisordtnal interaction between high and middle ability groups. High ability I ./. second semester was significantly better 
than the high ability students in the control group. The positions were reversed for the middle ability groups. 
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i experimental design - Analysis of Covariance used 
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School 1 ABILITY LEVELS FOR AMERICAN 



Level 1 






INDUSTRY AND CONTROL GROUPS 




Year: 1968-9 














Ability 

Pnrnm on /ore 


A.I. 








Control 






gl 


g2 




gl 


g2 


X 




114.1 


118.4 a 




118.4 a 


112.6 


S.D. 




7.4 


11.6 




11.6 


9.1 


V 




23 


21 




21 


15 


School B 
Level I 
Year: 1966-7 


Ability 

Paramenters 


Ati 








Control 






gl 


g2 g3 


g* 


gl 


g2 


T 




101.4 


102.6 a 109.3 


107.3 a 


107.3 


a 102.6 a 


S.D. 




12.3 


9.1 9.3 


5.7 


5.7 


9.1 


N 




20 


18 13 


14 


14 


18 



Year: 1967-11 



A bility 
Paramenters 


A.I. 






Control 




gl 


g2 


g3 


gl 


IT 


103.5 


104.1 


108.6 


105.4 


S.D. 


10.8 


7.4 


6.4 


10.5 


N 


18 


12 


12 


19 



Rotational design , first semester contrasts listed. 
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a 



Year: 1968-9 



1 hilitv 
Parameters 


u. 


Control 




g 1 


g* 


r 


103.4 a 


102.1 a 


S.D. 


10.1 


9.8 


/V 


20 


21 



School C 
Level 1 
Year: 1966-7 



Ability 

Parameters 


AJ. 






Control 




gl 


g2 


g3 


g 1 


g 2 


g$ 


X 


108.1 


110.7 


112.1 


112.1 


108.3 


108.5 


S.D. 


12.8 


11.0 


14.3 


13.1 


10.6 


9.7 


N 


18 


16 


22 


18 


18 


22 



Year : / 96 7-8 



Ability 

Parameters 


A. I. 




Control 




gl 


g2 


gl 


g 2 


r 


90.8 a 


91.3 a 


91.3 n 


90.8 a 


S.D. 


8.4 


11.1 


11.1 


* 

8.4 


Y 


20 


20 


20 

k 


20 



a Rotational design 
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Year: 1968 9 



A bilily 
Parameters 


AJ. 




Control 


_ 


gl 


M2 


Ml 


g2 


V 


101.5 a 


102.7 a 


102.7 a 


101.5 a 


S.D. 


10.8 


13.7 


13.7 


10.8 


V 


21 


21 


21 


21 



School D 
Level / 

Year: 1966-7 



Ability 

Parameters A. I. 


Control 




g 1 


g2 


gl 


g2 


M 3 


X 


107 A 


110.1 a 


106.7 


110.1 a 


108.3 


S.D. 


8.0 


9.5 


7.2 


9.5 


8.7 


S 


18 


18 


18 


18 


18 



Year: 1967-8 



Ability 

Parameters A. I. 


Control 




gl 


g2 


g 3 


g2 


g' 3 


X 


101.1 


108.5 a 


97.3 a 


108.5 a 


97.3 a 


S.D. 


12.4 


14.6 


14.0 


14.6 


14.0 


X 


17 


20 


23 


20 


23 



a 

Rotational design 
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VW: 1968-9 



1 bility 

Parameters A.L 


('on fro f 




gl 


g2 


g3 


gl 


A 


97.7 


101.3 


102.1 


102.8 


S.D. 


9.8 


10.5 


9.7 


11.1 


V 


23 


20 


22 


25 



Level 11 
Year: 1968-9 



A bility 

Parameters A.L 


Control 




g 1 


gi 




X 


104.3 


105.4 




S.D. 


6.2 


6.6 


V 18 


18 



School E 
Level I 
Year: 1966-7 



A bility 














Parameters 


A.L 








Control 






gl 


g2 


g3 


g4 


g } 


g2 


A 


276.9 


6 275.0 


28.5 


284.7 


266.2 


283.8 


S.D. 


8.1 


9.0 


8.8 


8.8 


10.8 


8.8 


V 


14 


14 


15 


15 


14 


12 




V f- 

to 
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STEP scores 



Year: 1967-8 



A bility 




Parameters A. I. 


Control 


_ e* 


gl 


V 1 06.7 


117.0 


S.D. 9.5 


15.0 


V 12 


14 



Year: 1968-9 



A bility 

— Parameters A.L 


Control 




g 1 


gi 


X 


98.1 


99.8 


S. D. 


12.7 


12.2 


V 35 


23 



School F 
Level II 
Year : 1966-7 



-4 bility 

Parameters A.L 


Control 




gi 




gi 


X 


269.7 6 




270.3 


S.D. 


10.8 




10.3 


1 V 


23 




19 



b STEP Scores 
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Year: 1967-8 



1 bility 










Parameters 


A.L 






Control 




gl 


g2 


& 


gl 


V 


267.6 b 


275.8 


270.3 


272.1 


S.D. 


11.1 


11.1 


9.3 


13.4 


/V 


17 


13 


8 


18 



b STEP Scores 
Year: 1968-9 



A bility 

Parameters A.l. 


Control 




gl 


gl 


V 


107.0 


106.2 


S.D. 


9.9 


6.7 


A' 26 


26 



School H 
Level I 
Year: 1966-7 



A bility 
Parameters 


A.l. 






Control 


, — 


gl 


g2 


g3 


gl 


g2 


X 


99.7 


104.2 


102.6 


102.6 a 


105. t 


S.D. 


9.7 


10.1 


10.1 


10.1 


8.8 


N 


21 


20 


20 


20 


20 



f I 



Rotational design 
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Year: 19678 



A hility 

Parameters A A. 


Control 


7 


gl 

103.1 n 


g2 

98.8 a 


fi'l 

99.0 a 


Hi 

99.0 r - 


82 

103.1 n 


83 

98.8 a 


S.D. 


10.6 


8.1 


8.1 


8.1 


10.6 


8.1 


V 


22 


18 


23 


23 


22 


18 



Year: 1968-9 



.1 hility 

Parameters A. I. 


Control 




Si 


S2 




7 


102.4 


108.0 




S.D. 


7.0 


12.9 




\ 


15 


15 





School I 
Level I 
Year: 1967-8 



Ability 

Parameters AA. 


Control 


7 


gl 

285.1 b 


S2 

281.6 b 


gl 

281.1 b 


g2 

276.5 b 




S.D. 


9.6 


8.6 


9.2 


8.9 




Y 


20 


20 


? — 

19 


19 





Year: 1968-9 



A bility 

Pnrnmotorv 


A A. 




Control 




gl 


g2 g3 




X 


280.8 b 


280. 1 b 278.0 b 




S.l). 


11.9 


10.9 6.7 




V 


12 


II 12 
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School K 
Level I 
Year: 196 6-7 



\hility 

IWameters {./. 


Control 


— gl g2 

v I0L4 a /0L2 a 


gl g2 

101.2 a W Li ° 


S - /A 9.9 10.1 


10.1 9.9 


v 50 51 


51 50 



Level II 
Year: 1967-8 



Ability 

Parameters A.I. 


Control 


— 8 1 g2 

V 101.6 a 101.8 a 


gl g2 

101.6 a 101.8 a 


S - D - 10.0 10.3 


10.0 10.3 


l 49 52 


49 52 



School L 
Level I 
Year: 1966-7 



Ability 

— Parameters A.I. 




— 


gl 


gl 


X 


101.2 c 


107.0 c 


S.D. 


11.3 


13.1 


v 25 


30 



e Quasi - experimental design 
a Rotational design 




J-51 

316 



Year: 196 7-8 



1 bilily 

Parameters 1 ./. 


(Ion (ml 




gl 




V 


103 




s.d. 


10.1 




^ 20 





Year: 1968-9 



Ability 

Parameters A.I. 


Control 




gl 


gl 


X 


108.2 


106.9 


S.D. 


8.9 


8.3 


N 


24 


20 



School M 
Level l 
Year: 1966-7 



Ability 

Parameters A. I. 


Control 




gl 


gl 


X 


108 


109.6 


S.D. 


8.0 


8.3 


V 29 


30 



Year: 1967-8 



A bility 

Parameters A f. 


Control 




gl 


gl 


X 


109.1 


105.3 


S.D. 


9.0 


7.4 


/V 


28 


27 



ERIC 
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School /V 
Level 1 
Year: / 96 7-8 



. 1 bill tv 

Parameters 1 




Control 




gl 


gll 


F 


110.6 


109.8 


S.D. 


8.1 


12.8 


/V 


20 


20 



School 0 • The set of ability scores for this school was not complete enough to provide 
the basis for meaningful ability parameters. 

School P 

Level l * 

Year: 19678 



Ability 

Parameters A. I. 




Control 




gl 


gl 


F 


113.4 


112.3 


S.D. 


6.9 


8.3 


V 


19 


17 



School R 
Level I 
Year: 1966-7 



A bility 

Parameters A. I. 


Control 




gl 


gl 


g2 


~X 


88.7 


89.7 


92.4 


S.D. 


9.9 


9.1 


9.7 


iV 


23 


19 


22 
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ANALYSIS OF ESSAY TEST DATA 






Introduction 



The secondary school courses in American Industry have been developed upon the basis of 
two general goals: (1) to build an understanding of the concepts which directly apply to indus- 

try, and (2) to develop the ability to solve problems related to industry (Face and Ftug, 1965). 
Inherent in the second goal was the assumption that a conceptual style of reasoning would lead 
to more effective problem solving. The essay test described in this paper was designed to gather 
data associated with the students' understanding of the concepts of industry and their ability to 
reason conceptually. 

Since knowledges, concepts, and styles of reasoning are internalized, they cannot be measured 
directly. In other words, a concept cannot be measured by placing a measuring device, such as 
a ruler, directly on it. Instead, a concept's characteristics have to be ascertained indirectly through 
its inferred influences on the responses given to various stimuli. Thus, the critical process encom- 
passed in designing the essay test was to determine the intervening role an understanding of the 
concepts of industry and a conceptual style of reasoning would play in structuring responses to 
various stimuli. From this information, appropriate test stimuli could be developed and logical 
answer keys written. 

These constructs are analyzed in greater detail in the ensuing paragraphs. The discussion 
touches on some of the influences of experience on perception, the affects of categorization on 
response, the role of concepts in thinking, and some of the parameters in concept attainment. A 
copy of the essay test is reproduced in Appendix K-A for those readers who may be interested 
in reviewing the test before reading on in the next section. 



Framework for the Essay Test 



American Industry courses are based on a structured body of knowledges comprised of con- 
cepts and their interrelationships. The source of this content is industry. A primary aim in 
evaluating these courses is to discern any significant changes that take place in the students en- 
rolled in American Industry. A point of departure for delineating the framework, therefore, is 
the influence of experience on perception and cognitive structure. 

A person forms a mental frame of reference based upon the important events and know- 
ledges in his repertoire of experiences. An artist perceives subtleties in form and texture. Others 
see a piece of pottery. A farmer views fertile soil and effective management during his trips 
through the countryside. The casual observer traveling the same roads sees tall corn and green 
fields. 

N. R. Hanson (1958) graphically demonstrated the effects of various frames of reference 
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with the use of the sketch shown in Figure 1. A bird watcher would probably see four birds 




However, the antelope hunter would identify them as antelope. And, rabbits would be observed 
by the ardent rabbit hunter. 

It is interesting to note that these three people would respond in a different manner to the 
same retinal image. Yet, three different interpretations are given. What could be the cause of 
this phenomenon? 



Notice that each of the people in the example has a special set of experiences. These ex- 
periences provide context for the illustration. Variation in the viewers' mental frame of reference 
causes them to perceive different animals in the same drawing. 



Experience with specific phenomena also increases the number and definitivesness of the re- 
sponses associated with these stimuli. The Eskimo, for example, has a large variety of words to 
describe snow. In contrast, there are no words for this type of weather in the vocabulary of the 
South Sea Islander. The former would be able to provide a unique description for a variety of 
snowy conditions. The latter would not understand snow. 

Bruner (1957) suggested that experience derives meaning from the mental category with which 
it is associated. The quality of what is perceived, according to Bruner, is a function of two 
factors: (1) the congruence between the stimulus and the category employed to interpret it, and 

(2) the degree to which this category has been developed. Categories are amplified through ex- 
perience and when the contents of a given category become too heterogeneous it is split into 
more homogeneous categories. 



Staats defined a process by which a word acquires meaning from concrete or abstract stim- 
uli and becomes the verbal label for a concept (1961). An ordered series of word responses are 
connected to this concept label. When the label is presented as a stimulus the ordered list of 
words forms the response. Another point made by Staats was that the composition and order 
within the list were the result of experience. Additional experiences may alter the order of the 
words in the response pattern or add new words to it. If this is the case, and it seems plausible, 
then the response pattern provides a valid gauge of the cognitive effects of experience. 
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To illustrate this, Staats' verbal hierarchy hypothesis will be applied to a set of free-associ- 
ation data. The stimulus word was research. Response patterns for seventh grade boys in Amer- 
ican Industry and in the control group are presented below. Only the first response of each 
student was tabulated and only those responses given by more than one student are reported. 

The numbers in the parentheses represent the proportion of the group giving each response. Var- 
iations in content and sequence are readily apparent. Responses made by the American Industry 
students focus on finding new information or knowledge. In contrast, responses made by the 
control group center on the discipline and people who do basic research. Since participation in 

American Industry 



finding, finding out (.27) 
^ ^Information (.21) 

/ ^ _ -Problem (.08) 
RESEARCH^ Z ~ 

- Study (.08) 
s v 'Solution (.06) 

'Market (.04) 



the American Industry course was the distinguishing characteristic between the two groups, the 
differences can logically be attributed to experiences in this course. 

Noble has suggested that meaning can be defined in terms of the responses produced for a 
given stimulus (1952). He also hypothesizes that meaning is a function of the number of re- 
sponces elicited by a given stimulus. Application of these ideas to the set of data just discussed 
would reveal that a significant difference in meaning exists between the two groups. Whereas the 
students in the control group view research as the province of science and scientist, the American 
Industry students conceive of it as a means to find answers for problems. 

A concept is more than a verbal label; and although the responses associated with this label 
may provide an index of its meaning as a word, the concept may function in a very different 
manner (Jenkins, 1966). Even though the American Industry students respond with action words 
to the stimulus word "research", they may react to problems and questions in the same manner 
as the students in the control group. A more detailed review of the nature of a concept will 
help to clarify this point. 

There are almost as many definitions of the term concept as there are psychologists, phil- 
osophers, and educators writing on the topic. However, most agree that a concept permits a 
person to judge a set of dissimilar objects or events as equivalent on the basis of one or more 
defining attributes. For example, all people who instruct students are labeled "teachers" by their 
students although they may vary in sex, age, size, and dress. In addition, when a person is per- 
ceived as a teacher he acquires the "characteristics" of a teacher that are encompassed in the 
student's concept of "teacher". For instance, most students-especially in college-expect a teacher 



Control Group 

Science (.20) 
Scientist (.08) 
Finding, finding 
out (.05) 

Development (.05) 
Problem (.05) 
Experiment (.05) 
Study (.05) 
Information (.03) 
Solution (.02) 
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to lecture. This expectation arises from experiences with teachers. 



w 



Bruner, Goodnow, and Austin (1956. p. 244) defined a concept as "...the network of in- 
ferences that are or may be set into play by an act of categorization." A basic factor in cor- 
relating a given stimulus with a specific concept is the ability to properly encode the stimulus. 

In other words, availability and selecnon of an appropriate verbal label facilitates the linking of 
a stimulus with the appropriate concept and triggers the remaining characteristics attached to this 
concept. These authors also noted that possession of a concept structures the sensory system to 
look for the concept's attributes in new stimuli. 



Three additional properties of concept attainment germane to the design of the essay test 
are: (1) the tendency for a concept to develop first on a concrete or functional basis and then 

mature to a symbolic relationship between objects or events. (2) the tendency for a concept to 
become more difficult to attain as the number of attributes in it increases, and (3) the preference 
for conjunctive concepts over the relational and disjunctive types. Bruner (1964) summarized man's 
attempts to comprehend a set of related events in the real-world as a process of moving from 
viewing them in terms of their physical properties to a comprehension of tha symbolic common- 
alties encompassed in the set. In some instances, he felt that the first stage could be by-passed 
if the learner had a highly developed cognitive system and the appropriate symbols. However, 
Bruner also cautioned that there was a definite risk in doing this. 

In general, as the number of relevant attributes in a concept increases, concept attainment 
becomes more difficult. Increasing the number of irrelevant attributes also adds to the complexity 
of the learning task. Posner (1964), for example, found that performance decreased as additional 
dimensions were added to the stimuli employed in his experiment. 



Several experimenters. Hunt and Hovland (1960), Conant and Trabasso (1964), and Wells 
(1963), have found that conjunctive concepts were easier to attain than disjunctive ones. In fact, 
their subjects were predisposed to search for conjunctive concepts. However, Wells found that 
training could change this search orientation to some degree. 



The ideas discussed in the previous pages are summarized in Table 1. 

In general, if the study of American Industry does produce unique learning, evidence of 
these effects should be found in the students' overt behavior. They should be able to produce 
more correct responses to questions based on industry, identify common elements in a group of 
stimuli, and provide more subtle answers than students who were not enrolled. The essay test 
designed to measure these behaviors is described in the next section. 



Essay Test 



This essay test is a part of the set of data gathering instruments employed by the Project 
to assess the outcomes of the study of American Industry. The essay format was selected to 
allow the student a greater latitude in responding. In addition, this compliments the objective 
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portion of the achievement test in which the student has to select one of the responses designed 
by the author of the test items. 

One of the basic objectives in designing the essay examination was to present test stimuli 
that were highly correlated with real-life situations in industry. Hence, pictorial stimuli were 
utilized in the first portion of the test. These stimuli are reproduced below. (A complete copy 
of the essay test is given in Appendix A.) 




Several criteria were established for selecting the stimuli. Their relation to the Project's 
structure of the knowledges of industry and to industrial activities was considered concurrently. 
Activities in the sketches had to represent valid industrial activities and fall within the domain o* 
the structure of industry established by the Project. Another important restraint was that several 
common elements or features had to run through the set of sketches. Moreover, these common- 
alities had to range from concrete to symbolic. It is rather obvious, for example, that each 
illustration contains one or more men. This is one of the concrete dimensions. A more symbolic 
dimension would be represented by the statement that all are using knowledge. 

Sketches were also selected that would portray four diverse enterprises. This provided an 
opportunity to assess the student's conception of industry and activities encompassed in it. 
Furthermore, the pictures had to be open ended so that a variety of interpretations could be made. 
This also furnished the chance for concepts to be derived at several levels of complexity. As 
noted above, man is a common element in each. More complex is the realization that each man 
is using tools to produce a product. 

Verbal questions were written for the lest part of the test. A different set was used during 
the two testing periods in which this test was employed. Both sets are given in Appendix K-A. 

In general, the questions in this part were designed to evaluate the students' general conception 
of industry and the relative importance they placed on the various concepts in the Project's 
structure of the knowledges of industry. 

As discussed in the previous section, extended experience in a specific realm should lead to 
a larger volume of responses and more definitive statements related to a representative subset of 
stimuli from the domain. Also, when these experiences are selected to convey a conceptual 
structure of knowledges the logical outcome is the ability to produce subtle answers and associate 
a given example with several elements in the structure. In other words, the student should h’ 
able to identify subtle concepts and relationships among concepts as well as being able to state 
the salient contents of the example. 



To assess these characteristics, the first five items were scored with two keys, fluency and 
saliency. Scores on the fluency scale indicate the number of correct responses listed for each 
question. A point was given for each correct response. Responses did not have to relate to the 
Project's structure of concepts; however, a response had to be unique to garner a point. Thus, 
for example, responses of "making a sign" and "painting a sign" made by the same individual 
were scored as one correct answer since they conveyed the same information. 



The saliency scale was developed to measure the perceptiveness displayed in the answer. At 
the most salient end were placed the concrete, simple, and obvious answers. "Man holding a 
tube", "painting a sign", and "men working" were statements contained in this category. At the 
least salient end of the scale fell the symbolic and complex answers such as "men using their 
skills to produce a product" and "men using skills and tools." To illustrate this point consider 
the content of the second question which is reproduced below. 




This depicts a man painting a sign-ct least on the concrete level it does. However, at a more 
perceptive level it shows a man using his knowledge of advertising in an attempt to improve 
sales. Other subtle ideas embedded in this picture are communication, competition, and profit. 



Each saliency scale was comprised of several categories. The c&tegories in each scale were 
numbered in consecutive order with the most salient category labelled "1" and the least salient 
assigned the largest number in the series. For example, if there were five categories in a specific 
saliency scale, the most salient category was represented by "1" and the least salient was repre- 
sented by "5". To determine the saliency score for a response, the test scorer isolated the least 
salient idea in the answer, read the numerical label for the category in which this idea fell, and 
recorded this number as the saliency score. (A complete description of the saliency keys is 
given in Appendix K-B). 

A third key, concept label, was designed to measure the degree of utilization of the concept 
labels by the responders. This was suggested by the work of Bruner and Staats reviewed earlier 
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